[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:32:07 dist-test-slave-mvvj krb5kdc[20417](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:32:07 dist-test-slave-mvvj krb5kdc[20417](info): set up 2 sockets
Aug 12 16:32:07 dist-test-slave-mvvj krb5kdc[20417](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 16:32:08.312575 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:32:08 dist-test-slave-mvvj krb5kdc[20417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:32:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:32:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:32:08.457190 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:38161
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:44885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:38161
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:08.546061 20432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:08.546314 20432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:08.546348 20432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:08.547641 20432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:08.547693 20432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:08.547708 20432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:08.547720 20432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:08.547732 20432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:32:08.549731 20432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:44885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:38161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:38161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:08.550161 20432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:08.550458 20432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:08.556686 20437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:08.556733 20432 server_base.cc:1034] running on GCE node
W20260812 16:32:08.556747 20443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:08.556686 20438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:08.557224 20432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:08.557659 20432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:08.558867 20432 hybrid_clock.cc:648] HybridClock initialized: now 1786552328558880 us; error 79 us; skew 500 ppm
Aug 12 16:32:08 dist-test-slave-mvvj krb5kdc[20417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552328, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:08.561586 20432 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:32:08.562106 20432 webserver.cc:460] Webserver started at http://127.19.238.126:35115/ using document root <none> and password file <none>
I20260812 16:32:08.562377 20432 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:08.562453 20432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:08.562615 20432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:08.563913 20432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "7cd33f78192f4a6f9a2141e206f6469e"
format_stamp: "Formatted at 2026-08-12 16:32:08 on dist-test-slave-mvvj"
server_key: "cdca89028a31a8da76643a59d0f0c853"
server_key_iv: "488b2323386b79cd93db5d0924380240"
server_key_version: "encryptionkey@0"
I20260812 16:32:08.564405 20432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "7cd33f78192f4a6f9a2141e206f6469e"
format_stamp: "Formatted at 2026-08-12 16:32:08 on dist-test-slave-mvvj"
server_key: "cdca89028a31a8da76643a59d0f0c853"
server_key_iv: "488b2323386b79cd93db5d0924380240"
server_key_version: "encryptionkey@0"
I20260812 16:32:08.565943 20432 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:32:08.566821 20450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:08.567180 20432 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:08.567308 20432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "7cd33f78192f4a6f9a2141e206f6469e"
format_stamp: "Formatted at 2026-08-12 16:32:08 on dist-test-slave-mvvj"
server_key: "cdca89028a31a8da76643a59d0f0c853"
server_key_iv: "488b2323386b79cd93db5d0924380240"
server_key_version: "encryptionkey@0"
I20260812 16:32:08.567404 20432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:08.599596 20432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:08.602044 20432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:08.602228 20432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:08.606878 20432 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:38161
I20260812 16:32:08.606916 20505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:38161 every 8 connection(s)
I20260812 16:32:08.607326 20432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:32:08.608601 20506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:08.611063 20506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e: Bootstrap starting.
I20260812 16:32:08.611701 20506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:08.612038 20506 log.cc:824] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:08.612908 20506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e: No bootstrap required, opened a new log
I20260812 16:32:08.613766 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 20432
I20260812 16:32:08.613911 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:32:08.614586 20409 external_mini_cluster.cc:1428] Setting key e7e0a328a01b82f05c4e1073fadae279
I20260812 16:32:08.614885 20506 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd33f78192f4a6f9a2141e206f6469e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38161 } }
I20260812 16:32:08.615034 20506 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:08.615064 20506 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd33f78192f4a6f9a2141e206f6469e, State: Initialized, Role: FOLLOWER
I20260812 16:32:08.615247 20506 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd33f78192f4a6f9a2141e206f6469e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38161 } }
I20260812 16:32:08.615414 20506 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:32:08.615475 20506 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:32:08.615537 20506 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:08.616672 20506 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd33f78192f4a6f9a2141e206f6469e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38161 } }
I20260812 16:32:08.616807 20506 leader_election.cc:302] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cd33f78192f4a6f9a2141e206f6469e; no voters:
I20260812 16:32:08.617022 20506 leader_election.cc:288] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:32:08.617048 20511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:08.617168 20511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e [term 1 LEADER]: Becoming Leader. State: Replica: 7cd33f78192f4a6f9a2141e206f6469e, State: Running, Role: LEADER
I20260812 16:32:08.617317 20506 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:08.617312 20511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd33f78192f4a6f9a2141e206f6469e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38161 } }
I20260812 16:32:08.617869 20512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cd33f78192f4a6f9a2141e206f6469e. Latest consensus state: current_term: 1 leader_uuid: "7cd33f78192f4a6f9a2141e206f6469e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd33f78192f4a6f9a2141e206f6469e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38161 } } }
I20260812 16:32:08.617976 20512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:08.618223 20513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cd33f78192f4a6f9a2141e206f6469e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd33f78192f4a6f9a2141e206f6469e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38161 } } }
I20260812 16:32:08.618592 20513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:08.618770 20518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:32:08.619385 20518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:32:08.624310 20518 catalog_manager.cc:1347] Generated new cluster ID: a7e590f8075b4081baee83cd83e1f508
I20260812 16:32:08.624437 20518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:32:08 dist-test-slave-mvvj krb5kdc[20417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:08.633278 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:08.615699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50414 (local address 127.19.238.126:38161)
0812 16:32:08.615961 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:08.615972 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:08.616241 (+ 269us) server_negotiation.cc:407] Connection header received
0812 16:32:08.618519 (+ 2278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:08.618526 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:08.618585 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:08.618748 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:08.620986 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:08.622407 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:08.623478 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:08.624154 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:08.624264 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:08.628891 (+ 4627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:08.628896 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:08.628897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:08.628911 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:08.631029 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:08.631572 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:08.631576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:08.631577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:08.631690 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:08.632038 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:08.632039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:08.632040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:08.632329 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:08.632496 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:08.632808 (+ 312us) server_negotiation.cc:299] Negotiation successful
0812 16:32:08.632955 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":154,"threads_started":1}
I20260812 16:32:08.633450 20518 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:32:08.634169 20518 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:08.637216 20518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cd33f78192f4a6f9a2141e206f6469e: Generated new TSK 0
I20260812 16:32:08.637424 20518 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
I20260812 16:32:08.671066 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:38161
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:44885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:08.799754 20534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:08.800005 20534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:08.800035 20534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:08.801357 20534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:08.801403 20534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:08.801433 20534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:32:08.802995 20534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:44885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:38161
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:08.803259 20534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:08.803463 20534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:08.808816 20539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:08.808866 20540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:08.808954 20542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:08.809178 20534 server_base.cc:1034] running on GCE node
I20260812 16:32:08.809338 20534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:08.809543 20534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:08.810704 20534 hybrid_clock.cc:648] HybridClock initialized: now 1786552328810688 us; error 29 us; skew 500 ppm
Aug 12 16:32:08 dist-test-slave-mvvj krb5kdc[20417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552328, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:08.813584 20534 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:32:08.814122 20534 webserver.cc:460] Webserver started at http://127.19.238.65:38627/ using document root <none> and password file <none>
I20260812 16:32:08.814368 20534 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:08.814440 20534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:08.814608 20534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:08.815714 20534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "6fa19571da174365babf0a5b5eecb3a6"
format_stamp: "Formatted at 2026-08-12 16:32:08 on dist-test-slave-mvvj"
server_key: "5ba9918ad8d30d3724fc93a82e8874e9"
server_key_iv: "fa6753256aa61841ca37f37a9c5d9d5f"
server_key_version: "encryptionkey@0"
I20260812 16:32:08.816187 20534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "6fa19571da174365babf0a5b5eecb3a6"
format_stamp: "Formatted at 2026-08-12 16:32:08 on dist-test-slave-mvvj"
server_key: "5ba9918ad8d30d3724fc93a82e8874e9"
server_key_iv: "fa6753256aa61841ca37f37a9c5d9d5f"
server_key_version: "encryptionkey@0"
I20260812 16:32:08.817742 20534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:32:08.818922 20549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:08.819120 20534 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:08.819219 20534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "6fa19571da174365babf0a5b5eecb3a6"
format_stamp: "Formatted at 2026-08-12 16:32:08 on dist-test-slave-mvvj"
server_key: "5ba9918ad8d30d3724fc93a82e8874e9"
server_key_iv: "fa6753256aa61841ca37f37a9c5d9d5f"
server_key_version: "encryptionkey@0"
I20260812 16:32:08.819360 20534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:08.848014 20534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:08.850462 20534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:08.850672 20534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:08.850957 20534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:08.851317 20534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:08.851377 20534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:08.851419 20534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:08.851464 20534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:08.858700 20534 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:33317
I20260812 16:32:08.858749 20678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:33317 every 8 connection(s)
I20260812 16:32:08.859130 20534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
Aug 12 16:32:08 dist-test-slave-mvvj krb5kdc[20417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552328, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:08.869144 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:08.859927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:50517 (local address 127.19.238.126:38161)
0812 16:32:08.860001 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:08.860004 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:08.860228 (+ 224us) server_negotiation.cc:407] Connection header received
0812 16:32:08.860726 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:08.860727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:08.860739 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:08.860790 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:08.861764 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:08.862275 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:08.863309 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:08.863552 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:08.863622 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:08.866199 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:08.866201 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:08.866202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:08.866210 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:08.867875 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:08.868262 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:08.868263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:08.868264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:08.868320 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:08.868547 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:08.868548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:08.868548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:08.868660 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:08.868711 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:08.868990 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 16:32:08.869075 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 16:32:08.869203 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 20534
I20260812 16:32:08.869292 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:32:08.869462 20683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:08.859920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:38161 (local address 127.19.238.65:50517)
0812 16:32:08.860092 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:08.860130 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:08.860631 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:08.860862 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:08.860865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:08.861476 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:08.861675 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:08.861680 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:08.862367 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:08.862368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:08.863194 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:08.863196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:08.863642 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:08.863643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:08.863751 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:08.864340 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:08.864348 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:08.866064 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:08.867942 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:08.867950 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:08.867951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:08.868191 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:08.868385 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:08.868385 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:08.868386 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:08.868490 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:08.868717 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:08.868722 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:08.868840 (+ 118us) client_negotiation.cc:769] Sending connection context
0812 16:32:08.868925 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 16:32:08.869163 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260812 16:32:08.869448 20409 external_mini_cluster.cc:1428] Setting key 7183bba0f2f9271d0ed6b98204a25ec3
I20260812 16:32:08.869998 20680 heartbeater.cc:347] Connected to a master server at 127.19.238.126:38161
I20260812 16:32:08.870127 20680 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:08.870337 20680 heartbeater.cc:511] Master 127.19.238.126:38161 requested a full tablet report, sending...
I20260812 16:32:08.870950 20466 ts_manager.cc:194] Registered new tserver with Master: 6fa19571da174365babf0a5b5eecb3a6 (127.19.238.65:33317)
I20260812 16:32:08.871678 20466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:50517
WARNING: no policy specified for kudu/127.19.238.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
I20260812 16:32:08.899552 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:38161
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:44885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:08.999775 20686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:09.000041 20686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:09.000073 20686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:09.001392 20686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:09.001456 20686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:09.001487 20686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:32:09.003078 20686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:44885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:38161
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:09.003363 20686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:09.003659 20686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:09.009367 20692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:09.009361 20693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:09.009366 20686 server_base.cc:1034] running on GCE node
W20260812 16:32:09.009480 20696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:09.009857 20686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:09.010144 20686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:09.011320 20686 hybrid_clock.cc:648] HybridClock initialized: now 1786552329011302 us; error 50 us; skew 500 ppm
Aug 12 16:32:09 dist-test-slave-mvvj krb5kdc[20417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552329, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:09.014549 20686 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:32:09.015079 20686 webserver.cc:460] Webserver started at http://127.19.238.66:39525/ using document root <none> and password file <none>
I20260812 16:32:09.015327 20686 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:09.015403 20686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:09.015586 20686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:09.016835 20686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "038754130f8e4b469e380aa73414553e"
format_stamp: "Formatted at 2026-08-12 16:32:09 on dist-test-slave-mvvj"
server_key: "27f5f68472d65a052ab5bcd565174b59"
server_key_iv: "bf1074f4b0e1398cc25e6a955b2b6d83"
server_key_version: "encryptionkey@0"
I20260812 16:32:09.017233 20686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "038754130f8e4b469e380aa73414553e"
format_stamp: "Formatted at 2026-08-12 16:32:09 on dist-test-slave-mvvj"
server_key: "27f5f68472d65a052ab5bcd565174b59"
server_key_iv: "bf1074f4b0e1398cc25e6a955b2b6d83"
server_key_version: "encryptionkey@0"
I20260812 16:32:09.018978 20686 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:09.019799 20704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:09.019999 20686 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:09.020104 20686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "038754130f8e4b469e380aa73414553e"
format_stamp: "Formatted at 2026-08-12 16:32:09 on dist-test-slave-mvvj"
server_key: "27f5f68472d65a052ab5bcd565174b59"
server_key_iv: "bf1074f4b0e1398cc25e6a955b2b6d83"
server_key_version: "encryptionkey@0"
I20260812 16:32:09.020200 20686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:09.036958 20686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:09.039300 20686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:09.039494 20686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:09.039763 20686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:09.040139 20686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:09.040198 20686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:09.040243 20686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:09.040267 20686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:09.046751 20686 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:38895
I20260812 16:32:09.046810 20825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:38895 every 8 connection(s)
I20260812 16:32:09.047195 20686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
Aug 12 16:32:09 dist-test-slave-mvvj krb5kdc[20417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552329, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:09.056622 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 20686
I20260812 16:32:09.056734 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:32:09.056931 20409 external_mini_cluster.cc:1428] Setting key 0ddfdcae58fc702f009f96ff4f3d6173
I20260812 16:32:09.058840 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:09.048047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:58757 (local address 127.19.238.126:38161)
0812 16:32:09.048144 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:09.048146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:09.048340 (+ 194us) server_negotiation.cc:407] Connection header received
0812 16:32:09.048874 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:09.048876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:09.048894 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:09.048952 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:09.049941 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:09.050485 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:09.051530 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:09.051727 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:09.051823 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:09.054681 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:09.054689 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:09.054689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:09.054697 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:09.057173 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:09.057689 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:09.057690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:09.057691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:09.057753 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:09.058082 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:09.058083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:09.058083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:09.058216 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:09.058293 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:09.058635 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 16:32:09.058746 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 16:32:09.059077 20828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:09.048013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:38161 (local address 127.19.238.66:58757)
0812 16:32:09.048210 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:09.048249 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:09.048746 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:09.049041 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:09.049044 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:09.049630 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:09.049830 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:09.049834 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:09.050593 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:09.050594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:09.051417 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:09.051419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:09.051885 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:09.051886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:09.052024 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:09.052675 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:09.052685 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:09.054340 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:09.057301 (+ 2961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:09.057314 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:09.057317 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:09.057597 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:09.057871 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:09.057872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:09.057873 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:09.058009 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:09.058296 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:09.058316 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:09.058491 (+ 175us) client_negotiation.cc:769] Sending connection context
0812 16:32:09.058613 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 16:32:09.058799 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":116,"threads_started":1}
I20260812 16:32:09.059587 20826 heartbeater.cc:347] Connected to a master server at 127.19.238.126:38161
I20260812 16:32:09.059715 20826 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:09.059962 20826 heartbeater.cc:511] Master 127.19.238.126:38161 requested a full tablet report, sending...
I20260812 16:32:09.060380 20466 ts_manager.cc:194] Registered new tserver with Master: 038754130f8e4b469e380aa73414553e (127.19.238.66:38895)
I20260812 16:32:09.060791 20466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:58757
WARNING: no policy specified for kudu/127.19.238.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
I20260812 16:32:09.088214 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:38161
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:44885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:09.175458 20833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:09.175676 20833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:09.175704 20833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:09.176916 20833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:09.176970 20833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:09.177000 20833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:32:09.178552 20833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:44885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:38161
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:09.178829 20833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:09.179083 20833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:09.184206 20841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:09.184201 20839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:09.184203 20838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:09.184497 20833 server_base.cc:1034] running on GCE node
I20260812 16:32:09.184711 20833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:09.185001 20833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:09.186204 20833 hybrid_clock.cc:648] HybridClock initialized: now 1786552329186195 us; error 40 us; skew 500 ppm
Aug 12 16:32:09 dist-test-slave-mvvj krb5kdc[20417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552329, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:09.189065 20833 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:32:09.189548 20833 webserver.cc:460] Webserver started at http://127.19.238.67:43717/ using document root <none> and password file <none>
I20260812 16:32:09.189802 20833 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:09.189873 20833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:09.190025 20833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:09.191114 20833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "752e7dca9b98447ba87af35693b983b6"
format_stamp: "Formatted at 2026-08-12 16:32:09 on dist-test-slave-mvvj"
server_key: "9bb5ad770dc0b166c27d2e90ed5a70d6"
server_key_iv: "03c3ace492d2f23c7c0b2a436f21a5a8"
server_key_version: "encryptionkey@0"
I20260812 16:32:09.191496 20833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "752e7dca9b98447ba87af35693b983b6"
format_stamp: "Formatted at 2026-08-12 16:32:09 on dist-test-slave-mvvj"
server_key: "9bb5ad770dc0b166c27d2e90ed5a70d6"
server_key_iv: "03c3ace492d2f23c7c0b2a436f21a5a8"
server_key_version: "encryptionkey@0"
I20260812 16:32:09.192993 20833 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:32:09.193835 20850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:09.194062 20833 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:09.194165 20833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "752e7dca9b98447ba87af35693b983b6"
format_stamp: "Formatted at 2026-08-12 16:32:09 on dist-test-slave-mvvj"
server_key: "9bb5ad770dc0b166c27d2e90ed5a70d6"
server_key_iv: "03c3ace492d2f23c7c0b2a436f21a5a8"
server_key_version: "encryptionkey@0"
I20260812 16:32:09.194248 20833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:09.206789 20833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:09.209115 20833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:09.209292 20833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:09.209563 20833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:09.209916 20833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:09.209959 20833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:09.209996 20833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:09.210027 20833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:09.216248 20833 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:35401
I20260812 16:32:09.216372 20977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:35401 every 8 connection(s)
I20260812 16:32:09.216656 20833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
Aug 12 16:32:09 dist-test-slave-mvvj krb5kdc[20417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552329, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:09.224527 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 20833
I20260812 16:32:09.224629 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:32:09.224821 20409 external_mini_cluster.cc:1428] Setting key b19f875d27ea9b4ce85704bac7705afc
I20260812 16:32:09.227257 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:09.217561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:54467 (local address 127.19.238.126:38161)
0812 16:32:09.217644 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:09.217647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:09.217852 (+ 205us) server_negotiation.cc:407] Connection header received
0812 16:32:09.218553 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:09.218554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:09.218568 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:09.218631 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:09.219688 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:09.220273 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:09.221376 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:09.221579 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:09.221654 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:09.224198 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:09.224200 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:09.224200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:09.224208 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:09.225815 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:09.226247 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:09.226248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:09.226248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:09.226313 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:09.226606 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:09.226607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:09.226607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:09.226725 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:09.226802 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:09.227078 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 16:32:09.227160 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 16:32:09.227496 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:09.217524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:38161 (local address 127.19.238.67:54467)
0812 16:32:09.217710 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:09.217748 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:09.218432 (+ 684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:09.218720 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:09.218728 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:09.219386 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:09.219588 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:09.219593 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:09.220427 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:09.220428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:09.221237 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:09.221240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:09.221677 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:09.221678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:09.221875 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:09.222476 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:09.222490 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:09.224094 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:09.225921 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:09.225928 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:09.225930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:09.226166 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:09.226428 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:09.226429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:09.226429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:09.226527 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:09.226794 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:09.226800 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:09.226945 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 16:32:09.227056 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 16:32:09.227229 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":76,"threads_started":1}
I20260812 16:32:09.228044 20978 heartbeater.cc:347] Connected to a master server at 127.19.238.126:38161
I20260812 16:32:09.228139 20978 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:09.228286 20978 heartbeater.cc:511] Master 127.19.238.126:38161 requested a full tablet report, sending...
I20260812 16:32:09.228643 20466 ts_manager.cc:194] Registered new tserver with Master: 752e7dca9b98447ba87af35693b983b6 (127.19.238.67:35401)
I20260812 16:32:09.228997 20466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:54467
I20260812 16:32:09.236694 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 16:32:09 dist-test-slave-mvvj krb5kdc[20417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552329, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 16:32:09 dist-test-slave-mvvj krb5kdc[20417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552329, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.238.65@KRBTEST.COM
I20260812 16:32:09.264226 20986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:09.254857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37982 (local address 127.19.238.65:33317)
0812 16:32:09.255040 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:09.255046 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:09.255061 (+ 15us) server_negotiation.cc:407] Connection header received
0812 16:32:09.255092 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:09.255094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:09.255110 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:09.255181 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:09.255804 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:09.256640 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:09.257388 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:09.257716 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:09.257780 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:09.260981 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:09.261000 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:09.261000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:09.261008 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:09.262616 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:09.263045 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:09.263048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:09.263049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:09.263119 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:09.263338 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:09.263339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:09.263339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:09.263609 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:09.263825 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:09.263974 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 16:32:09.264077 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
W20260812 16:32:09.264662 20578 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37982
Aug 12 16:32:09 dist-test-slave-mvvj krb5kdc[20417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 16:32:09 dist-test-slave-mvvj krb5kdc[20417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.65@KRBTEST.COM
I20260812 16:32:09.289289 20986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:09.281582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37996 (local address 127.19.238.65:33317)
0812 16:32:09.281679 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:09.281681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:09.281697 (+ 16us) server_negotiation.cc:407] Connection header received
0812 16:32:09.281780 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:09.281781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:09.281795 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:09.281849 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:09.282403 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:09.282906 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:09.283615 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:09.283810 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:09.283951 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:09.286373 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:09.286374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:09.286374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:09.286384 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:09.287824 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:09.288231 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:09.288232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:09.288232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:09.288292 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:09.288611 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:09.288612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:09.288612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:09.288734 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:09.288866 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:09.289021 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 16:32:09.289153 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 16:32:09.289978 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 20534
I20260812 16:32:09.303160 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 20686
I20260812 16:32:09.313170 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 20833
I20260812 16:32:09.320108 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 20432
2026-08-12T16:32:09Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2149 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:32:09 dist-test-slave-mvvj krb5kdc[20995](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:32:09 dist-test-slave-mvvj krb5kdc[20995](info): set up 2 sockets
Aug 12 16:32:09 dist-test-slave-mvvj krb5kdc[20995](info): commencing operation
krb5kdc: starting...
W20260812 16:32:10.454797 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:32:10 dist-test-slave-mvvj krb5kdc[20995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:32:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:32:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:32:10.591794 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:37039
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:45149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:37039
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:10.676390 21008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:10.676618 21008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:10.676641 21008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:10.677929 21008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:10.677981 21008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:10.677996 21008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:10.678009 21008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:10.678025 21008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:32:10.679607 21008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:45149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:37039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:37039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:10.679924 21008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:10.680181 21008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 16:32:10.685456 21008 server_base.cc:1034] running on GCE node
W20260812 16:32:10.685432 21014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:10.685472 21016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:10.685423 21013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:10.685851 21008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:10.686118 21008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:10.687281 21008 hybrid_clock.cc:648] HybridClock initialized: now 1786552330687260 us; error 43 us; skew 500 ppm
Aug 12 16:32:10 dist-test-slave-mvvj krb5kdc[20995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552330, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:10.690061 21008 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:32:10.690563 21008 webserver.cc:460] Webserver started at http://127.19.238.126:43783/ using document root <none> and password file <none>
I20260812 16:32:10.690804 21008 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:10.690896 21008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:10.691051 21008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:10.692292 21008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "7c80927f1e4d4b58b35fd55e51787af0"
format_stamp: "Formatted at 2026-08-12 16:32:10 on dist-test-slave-mvvj"
server_key: "bfc0ec9fa9fceb936a82c344a03e181b"
server_key_iv: "28333d6b56df523141febc3030550271"
server_key_version: "encryptionkey@0"
I20260812 16:32:10.692693 21008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "7c80927f1e4d4b58b35fd55e51787af0"
format_stamp: "Formatted at 2026-08-12 16:32:10 on dist-test-slave-mvvj"
server_key: "bfc0ec9fa9fceb936a82c344a03e181b"
server_key_iv: "28333d6b56df523141febc3030550271"
server_key_version: "encryptionkey@0"
I20260812 16:32:10.694271 21008 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:32:10.695204 21023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:10.695539 21008 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:10.695662 21008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "7c80927f1e4d4b58b35fd55e51787af0"
format_stamp: "Formatted at 2026-08-12 16:32:10 on dist-test-slave-mvvj"
server_key: "bfc0ec9fa9fceb936a82c344a03e181b"
server_key_iv: "28333d6b56df523141febc3030550271"
server_key_version: "encryptionkey@0"
I20260812 16:32:10.695746 21008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:10.709527 21008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:10.712162 21008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:10.712352 21008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:10.716872 21008 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:37039
I20260812 16:32:10.716943 21079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:37039 every 8 connection(s)
I20260812 16:32:10.717351 21008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:32:10.717658 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 21008
I20260812 16:32:10.717760 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:32:10.717957 20409 external_mini_cluster.cc:1428] Setting key 95eac6b583d6c1b940a8e96e8a143231
I20260812 16:32:10.718658 21080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:10.722220 21080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0: Bootstrap starting.
I20260812 16:32:10.722985 21080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:10.723383 21080 log.cc:824] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:10.724396 21080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0: No bootstrap required, opened a new log
Aug 12 16:32:10 dist-test-slave-mvvj krb5kdc[20995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:10.727227 21080 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c80927f1e4d4b58b35fd55e51787af0" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37039 } }
I20260812 16:32:10.727459 21080 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:10.727542 21080 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c80927f1e4d4b58b35fd55e51787af0, State: Initialized, Role: FOLLOWER
I20260812 16:32:10.727715 21080 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c80927f1e4d4b58b35fd55e51787af0" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37039 } }
I20260812 16:32:10.727809 21080 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:32:10.727883 21080 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:32:10.727941 21080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:10.728585 21080 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c80927f1e4d4b58b35fd55e51787af0" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37039 } }
I20260812 16:32:10.728717 21080 leader_election.cc:302] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c80927f1e4d4b58b35fd55e51787af0; no voters:
I20260812 16:32:10.728952 21080 leader_election.cc:288] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:32:10.728992 21086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:10.729261 21086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0 [term 1 LEADER]: Becoming Leader. State: Replica: 7c80927f1e4d4b58b35fd55e51787af0, State: Running, Role: LEADER
I20260812 16:32:10.729411 21086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c80927f1e4d4b58b35fd55e51787af0" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37039 } }
I20260812 16:32:10.729393 21080 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:10.729949 21090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c80927f1e4d4b58b35fd55e51787af0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c80927f1e4d4b58b35fd55e51787af0" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37039 } } }
I20260812 16:32:10.730077 21090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:10.730276 21084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:10.718555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46550 (local address 127.19.238.126:37039)
0812 16:32:10.718744 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:10.718754 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:10.718797 (+ 43us) server_negotiation.cc:407] Connection header received
0812 16:32:10.719279 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:10.719286 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:10.719338 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:10.719466 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:10.720488 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:10.721417 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:10.722569 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:10.722987 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:10.723120 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:10.726094 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:10.726098 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:10.726099 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:10.726110 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:10.728332 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:10.728746 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:10.728749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:10.728750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:10.728936 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:10.729169 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:10.729170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:10.729170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:10.729503 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:10.729641 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:10.729842 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 16:32:10.730014 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
I20260812 16:32:10.730352 21093 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:32:10.730973 21093 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:32:10.729935 21091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c80927f1e4d4b58b35fd55e51787af0. Latest consensus state: current_term: 1 leader_uuid: "7c80927f1e4d4b58b35fd55e51787af0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c80927f1e4d4b58b35fd55e51787af0" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37039 } } }
I20260812 16:32:10.731613 21091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:10.732924 21093 catalog_manager.cc:1347] Generated new cluster ID: 0b453e66ded14172834c8284a5134458
I20260812 16:32:10.732983 21093 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:32:10.741169 21093 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:32:10.741958 21093 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:10.745757 21093 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c80927f1e4d4b58b35fd55e51787af0: Generated new TSK 0
I20260812 16:32:10.745962 21093 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
I20260812 16:32:10.783032 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:37039
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:45149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:10.869920 21112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:10.870149 21112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:10.870182 21112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:10.871318 21112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:10.871358 21112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:10.871385 21112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:32:10.872892 21112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:45149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:37039
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:10.873099 21112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:10.873292 21112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:10.878535 21117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:10.878561 21120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:10.878535 21118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:10.879010 21112 server_base.cc:1034] running on GCE node
I20260812 16:32:10.879202 21112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:10.879406 21112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:10.880612 21112 hybrid_clock.cc:648] HybridClock initialized: now 1786552330880597 us; error 25 us; skew 500 ppm
Aug 12 16:32:10 dist-test-slave-mvvj krb5kdc[20995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552330, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:10.883332 21112 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:32:10.883812 21112 webserver.cc:460] Webserver started at http://127.19.238.65:46395/ using document root <none> and password file <none>
I20260812 16:32:10.884126 21112 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:10.884198 21112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:10.884352 21112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:10.885433 21112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "b2b690173e9947beb7fa84eea32bb595"
format_stamp: "Formatted at 2026-08-12 16:32:10 on dist-test-slave-mvvj"
server_key: "2853db1202d225f2c4cac64068d22557"
server_key_iv: "6997e4b8f0bfe15cf535443b42cba6ba"
server_key_version: "encryptionkey@0"
I20260812 16:32:10.885823 21112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "b2b690173e9947beb7fa84eea32bb595"
format_stamp: "Formatted at 2026-08-12 16:32:10 on dist-test-slave-mvvj"
server_key: "2853db1202d225f2c4cac64068d22557"
server_key_iv: "6997e4b8f0bfe15cf535443b42cba6ba"
server_key_version: "encryptionkey@0"
I20260812 16:32:10.887249 21112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:32:10.888237 21128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:10.888528 21112 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:10.888638 21112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "b2b690173e9947beb7fa84eea32bb595"
format_stamp: "Formatted at 2026-08-12 16:32:10 on dist-test-slave-mvvj"
server_key: "2853db1202d225f2c4cac64068d22557"
server_key_iv: "6997e4b8f0bfe15cf535443b42cba6ba"
server_key_version: "encryptionkey@0"
I20260812 16:32:10.888726 21112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:10.902498 21112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:10.905010 21112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:10.905193 21112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:10.905465 21112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:10.905814 21112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:10.905871 21112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:10.905913 21112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:10.905963 21112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:10.912187 21112 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:42653
I20260812 16:32:10.912331 21252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:42653 every 8 connection(s)
I20260812 16:32:10.912642 21112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:32:10.919086 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 21112
I20260812 16:32:10.919224 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:32:10.919402 20409 external_mini_cluster.cc:1428] Setting key 0279f13828f80fd8eee0ec6a42f80f7d
Aug 12 16:32:10 dist-test-slave-mvvj krb5kdc[20995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552330, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:10.924057 21084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:10.913524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:36973 (local address 127.19.238.126:37039)
0812 16:32:10.913602 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:10.913604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:10.913822 (+ 218us) server_negotiation.cc:407] Connection header received
0812 16:32:10.914924 (+ 1102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:10.914925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:10.914942 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:10.915035 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:10.915798 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:10.916368 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:10.917450 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:10.917664 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:10.917754 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:10.920603 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:10.920605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:10.920606 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:10.920615 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:10.922288 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:10.922872 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:10.922873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:10.922874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:10.922931 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:10.923262 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:10.923264 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:10.923264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:10.923411 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:10.923502 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:10.923825 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 16:32:10.923954 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 16:32:10.924269 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:10.913475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37039 (local address 127.19.238.65:36973)
0812 16:32:10.913639 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:10.913684 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:10.914755 (+ 1071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:10.915162 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:10.915166 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:10.915474 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:10.915679 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:10.915684 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:10.916479 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:10.916480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:10.917338 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:10.917340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:10.917772 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:10.917773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:10.917906 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:10.918466 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:10.918477 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:10.920475 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:10.922383 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:10.922392 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:10.922395 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:10.922779 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:10.923016 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:10.923017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:10.923017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:10.923180 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:10.923493 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:10.923501 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:10.923666 (+ 165us) client_negotiation.cc:769] Sending connection context
0812 16:32:10.923786 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 16:32:10.924006 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260812 16:32:10.924813 21253 heartbeater.cc:347] Connected to a master server at 127.19.238.126:37039
I20260812 16:32:10.924918 21253 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:10.925143 21253 heartbeater.cc:511] Master 127.19.238.126:37039 requested a full tablet report, sending...
I20260812 16:32:10.925714 21043 ts_manager.cc:194] Registered new tserver with Master: b2b690173e9947beb7fa84eea32bb595 (127.19.238.65:42653)
I20260812 16:32:10.926419 21043 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:36973
WARNING: no policy specified for kudu/127.19.238.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
I20260812 16:32:10.951202 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:37039
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:45149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:11.038404 21260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:11.038637 21260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:11.038663 21260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:11.039788 21260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:11.039825 21260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:11.039875 21260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:32:11.041343 21260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:45149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:37039
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:11.041556 21260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:11.041769 21260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:11.047168 21266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:11.047291 21260 server_base.cc:1034] running on GCE node
W20260812 16:32:11.047108 21267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:11.047082 21269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:11.047717 21260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:11.048034 21260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:11.049206 21260 hybrid_clock.cc:648] HybridClock initialized: now 1786552331049203 us; error 38 us; skew 500 ppm
Aug 12 16:32:11 dist-test-slave-mvvj krb5kdc[20995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552331, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:11.051939 21260 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:32:11.052461 21260 webserver.cc:460] Webserver started at http://127.19.238.66:34319/ using document root <none> and password file <none>
I20260812 16:32:11.052716 21260 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:11.052788 21260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:11.052944 21260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:11.054080 21260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "fbbe5b2252174173af0a60365055bef2"
format_stamp: "Formatted at 2026-08-12 16:32:11 on dist-test-slave-mvvj"
server_key: "7fa5be415e66712af567136234eff0b2"
server_key_iv: "7e260c63153f07d6493f89fcb64cf774"
server_key_version: "encryptionkey@0"
I20260812 16:32:11.054517 21260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "fbbe5b2252174173af0a60365055bef2"
format_stamp: "Formatted at 2026-08-12 16:32:11 on dist-test-slave-mvvj"
server_key: "7fa5be415e66712af567136234eff0b2"
server_key_iv: "7e260c63153f07d6493f89fcb64cf774"
server_key_version: "encryptionkey@0"
I20260812 16:32:11.056071 21260 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:32:11.056911 21276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:11.057087 21260 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:11.057199 21260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "fbbe5b2252174173af0a60365055bef2"
format_stamp: "Formatted at 2026-08-12 16:32:11 on dist-test-slave-mvvj"
server_key: "7fa5be415e66712af567136234eff0b2"
server_key_iv: "7e260c63153f07d6493f89fcb64cf774"
server_key_version: "encryptionkey@0"
I20260812 16:32:11.057282 21260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:11.065274 21260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:11.067472 21260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:11.067656 21260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:11.067967 21260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:11.068306 21260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:11.068367 21260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:11.068413 21260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:11.068439 21260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:11.074496 21260 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:46173
I20260812 16:32:11.074540 21402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:46173 every 8 connection(s)
I20260812 16:32:11.074884 21260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:32:11.077363 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 21260
I20260812 16:32:11.077461 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:32:11.077625 20409 external_mini_cluster.cc:1428] Setting key 558f946b744c5b00df4d39481ec5da98
Aug 12 16:32:11 dist-test-slave-mvvj krb5kdc[20995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552331, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:11.086669 21084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:11.075702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:45897 (local address 127.19.238.126:37039)
0812 16:32:11.075785 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:11.075788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:11.076045 (+ 257us) server_negotiation.cc:407] Connection header received
0812 16:32:11.077030 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:11.077032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:11.077043 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:11.077120 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:11.078054 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.078700 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.080276 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.080610 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.080694 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:11.083307 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:11.083309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:11.083309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:11.083318 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:11.085168 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:11.085610 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:11.085611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:11.085612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:11.085683 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:11.085953 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:11.085954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:11.085955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:11.086115 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:11.086195 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:11.086459 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 16:32:11.086565 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 16:32:11.086716 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:11.075650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37039 (local address 127.19.238.66:45897)
0812 16:32:11.075808 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:11.075939 (+ 131us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:11.076869 (+ 930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:11.077215 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:11.077219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:11.077626 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:11.077941 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:11.077948 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.078807 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.078808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:11.080174 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:11.080178 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.080707 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.080708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:11.080814 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:11.081390 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:11.081398 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:11.083204 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:11.085264 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:11.085272 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:11.085273 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:11.085522 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:11.085757 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:11.085758 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:11.085758 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:11.085879 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:11.086189 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:11.086194 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:11.086318 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 16:32:11.086406 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 16:32:11.086538 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260812 16:32:11.087354 21403 heartbeater.cc:347] Connected to a master server at 127.19.238.126:37039
I20260812 16:32:11.087489 21403 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:11.087692 21403 heartbeater.cc:511] Master 127.19.238.126:37039 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.238.67@KRBTEST.COM; defaulting to no policy
I20260812 16:32:11.088218 21043 ts_manager.cc:194] Registered new tserver with Master: fbbe5b2252174173af0a60365055bef2 (127.19.238.66:46173)
I20260812 16:32:11.088644 21043 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:45897
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
I20260812 16:32:11.107261 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:37039
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:45149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:11.191220 21408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:11.191488 21408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:11.191532 21408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:11.192942 21408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:11.193012 21408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:11.193060 21408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:32:11.194656 21408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:45149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:37039
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:11.194940 21408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:11.195183 21408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:11.200788 21416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:11.200806 21414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:11.200821 21413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:11.200963 21408 server_base.cc:1034] running on GCE node
I20260812 16:32:11.201186 21408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:11.201402 21408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:11.202572 21408 hybrid_clock.cc:648] HybridClock initialized: now 1786552331202558 us; error 30 us; skew 500 ppm
Aug 12 16:32:11 dist-test-slave-mvvj krb5kdc[20995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552331, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:11.205420 21408 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:32:11.205926 21408 webserver.cc:460] Webserver started at http://127.19.238.67:33433/ using document root <none> and password file <none>
I20260812 16:32:11.206212 21408 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:11.206287 21408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:11.206418 21408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:11.207476 21408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "bad95cef338d4797986b7a943ba43784"
format_stamp: "Formatted at 2026-08-12 16:32:11 on dist-test-slave-mvvj"
server_key: "58d7cd6d7ffd60261657c01696dbdf9d"
server_key_iv: "71e5647e15bab820bfd06562a68dce89"
server_key_version: "encryptionkey@0"
I20260812 16:32:11.207789 21408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "bad95cef338d4797986b7a943ba43784"
format_stamp: "Formatted at 2026-08-12 16:32:11 on dist-test-slave-mvvj"
server_key: "58d7cd6d7ffd60261657c01696dbdf9d"
server_key_iv: "71e5647e15bab820bfd06562a68dce89"
server_key_version: "encryptionkey@0"
I20260812 16:32:11.209317 21408 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:11.210273 21424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:11.210548 21408 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:11.210654 21408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "bad95cef338d4797986b7a943ba43784"
format_stamp: "Formatted at 2026-08-12 16:32:11 on dist-test-slave-mvvj"
server_key: "58d7cd6d7ffd60261657c01696dbdf9d"
server_key_iv: "71e5647e15bab820bfd06562a68dce89"
server_key_version: "encryptionkey@0"
I20260812 16:32:11.210770 21408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:11.221449 21408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:11.224138 21408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:11.224331 21408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:11.224629 21408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:11.225008 21408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:11.225047 21408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:11.225075 21408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:11.225088 21408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:11.232255 21408 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:38939
I20260812 16:32:11.232295 21554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:38939 every 8 connection(s)
I20260812 16:32:11.232699 21408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:32:11.233170 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 21408
I20260812 16:32:11.233273 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:32:11.233448 20409 external_mini_cluster.cc:1428] Setting key 72fde74755d74a0c3c7dea3cbcf1f5b7
Aug 12 16:32:11 dist-test-slave-mvvj krb5kdc[20995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552331, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:11.242905 21084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:11.233633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:40895 (local address 127.19.238.126:37039)
0812 16:32:11.233708 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:11.233711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:11.234039 (+ 328us) server_negotiation.cc:407] Connection header received
0812 16:32:11.235253 (+ 1214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:11.235254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:11.235264 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:11.235308 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:11.236052 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.236553 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.237584 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.237774 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.237857 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:11.240220 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:11.240221 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:11.240221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:11.240228 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:11.241469 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:11.241932 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:11.241933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:11.241933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:11.242004 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:11.242323 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:11.242324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:11.242325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:11.242451 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:11.242520 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:11.242763 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 16:32:11.242831 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 16:32:11.243109 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:11.233616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37039 (local address 127.19.238.67:40895)
0812 16:32:11.233891 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:11.233939 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:11.235154 (+ 1215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:11.235382 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:11.235386 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:11.235657 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:11.235971 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:11.235977 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.236658 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.236660 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:11.237484 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:11.237487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.237888 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.237890 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:11.237994 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:11.238597 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:11.238606 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:11.240122 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:11.241554 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:11.241561 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:11.241562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:11.241828 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:11.242110 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:11.242111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:11.242112 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:11.242224 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:11.242523 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:11.242528 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:11.242643 (+ 115us) client_negotiation.cc:769] Sending connection context
0812 16:32:11.242728 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 16:32:11.242864 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":123,"threads_started":1}
I20260812 16:32:11.243896 21555 heartbeater.cc:347] Connected to a master server at 127.19.238.126:37039
I20260812 16:32:11.244112 21555 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:11.244328 21555 heartbeater.cc:511] Master 127.19.238.126:37039 requested a full tablet report, sending...
I20260812 16:32:11.244910 21043 ts_manager.cc:194] Registered new tserver with Master: bad95cef338d4797986b7a943ba43784 (127.19.238.67:38939)
I20260812 16:32:11.245433 21043 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:40895
I20260812 16:32:11.245563 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:32:11.254487 21084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:11.247188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46552 (local address 127.19.238.126:37039)
0812 16:32:11.247270 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:11.247272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:11.248418 (+ 1146us) server_negotiation.cc:407] Connection header received
0812 16:32:11.248513 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:11.248514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:11.248524 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:11.248597 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:11.249172 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.249649 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.250419 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.250616 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.250691 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:11.252035 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:11.252036 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:11.252037 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:11.252044 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:11.253301 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:11.253653 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:11.253654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:11.253654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:11.253711 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:11.253915 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:11.253916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:11.253917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:11.254078 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:11.254194 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:11.254300 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 16:32:11.254380 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 16:32:11.256855 21043 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:32:11.258515 21043 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:32:11.266059 21573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:11.261746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:42653 (local address 127.0.0.1:47736)
0812 16:32:11.261868 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:11.261885 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:11.261930 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:11.262365 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:11.262366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:11.262367 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:11.262442 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:11.262444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.263737 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.263738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:11.265044 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:11.265047 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.265753 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.265754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:11.265834 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:11.265848 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 16:32:11.265918 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 16:32:11.265972 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":77,"thread_start_us":86,"threads_started":1}
I20260812 16:32:11.266069 21569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:11.261538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:38939 (local address 127.0.0.1:46674)
0812 16:32:11.261824 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:11.261836 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:11.261900 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:11.262691 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:11.262692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:11.262694 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:11.262739 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:11.262741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.263964 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.263966 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:11.264913 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:11.264916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.265894 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.265895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:11.265950 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:11.265953 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:32:11.265988 (+ 35us) client_negotiation.cc:240] Negotiation successful
0812 16:32:11.266018 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":95,"threads_started":1}
I20260812 16:32:11.266350 21568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:11.261508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:46173 (local address 127.0.0.1:38584)
0812 16:32:11.261727 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:11.261773 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:11.261851 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:11.262284 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:11.262287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:11.262290 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:11.262364 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:11.262367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.263513 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.263514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:11.265045 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:11.265047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.266068 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.266069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:11.266136 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:11.266140 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:11.266199 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 16:32:11.266254 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":50,"thread_start_us":78,"threads_started":1}
I20260812 16:32:11.266469 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:11.262020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47736 (local address 127.19.238.65:42653)
0812 16:32:11.262211 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:11.262215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:11.262224 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:32:11.262264 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:11.262266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:11.262280 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:11.262359 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:11.262512 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.263619 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.265144 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.265641 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.265745 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:11.265810 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:11.266298 (+ 488us) server_negotiation.cc:299] Negotiation successful
0812 16:32:11.266351 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":105,"threads_started":1}
I20260812 16:32:11.266619 21571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:11.261629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38584 (local address 127.19.238.66:46173)
0812 16:32:11.262022 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:11.262027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:11.262037 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:11.262081 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:11.262083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:11.262101 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:11.262179 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:11.262474 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.263387 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.265141 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.265906 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.265973 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:11.266029 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:11.266478 (+ 449us) server_negotiation.cc:299] Negotiation successful
0812 16:32:11.266524 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":103,"threads_started":1}
I20260812 16:32:11.267026 21572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:11.261643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46674 (local address 127.19.238.67:38939)
0812 16:32:11.262524 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:11.262527 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:11.262537 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:11.262577 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:11.262579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:11.262594 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:11.262680 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:11.262816 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.263876 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.265019 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.265811 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.265897 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:11.265972 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:11.266015 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 16:32:11.266065 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":840,"thread_start_us":93,"threads_started":1}
I20260812 16:32:11.267235 21178 tablet_service.cc:1467] Processing CreateTablet for tablet e225de9e71de4760a6d8a9a97b373eaa (DEFAULT_TABLE table=test-table [id=001bdb41f2bf48b096e01a7329fc694a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:11.267385 21329 tablet_service.cc:1467] Processing CreateTablet for tablet e225de9e71de4760a6d8a9a97b373eaa (DEFAULT_TABLE table=test-table [id=001bdb41f2bf48b096e01a7329fc694a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:11.267632 21178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e225de9e71de4760a6d8a9a97b373eaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:11.267675 21329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e225de9e71de4760a6d8a9a97b373eaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:11.267701 21479 tablet_service.cc:1467] Processing CreateTablet for tablet e225de9e71de4760a6d8a9a97b373eaa (DEFAULT_TABLE table=test-table [id=001bdb41f2bf48b096e01a7329fc694a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:11.268036 21479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e225de9e71de4760a6d8a9a97b373eaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:11.270474 21575 tablet_bootstrap.cc:492] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2: Bootstrap starting.
I20260812 16:32:11.271142 21576 tablet_bootstrap.cc:492] T e225de9e71de4760a6d8a9a97b373eaa P b2b690173e9947beb7fa84eea32bb595: Bootstrap starting.
I20260812 16:32:11.271502 21575 tablet_bootstrap.cc:654] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:11.271543 21577 tablet_bootstrap.cc:492] T e225de9e71de4760a6d8a9a97b373eaa P bad95cef338d4797986b7a943ba43784: Bootstrap starting.
I20260812 16:32:11.271962 21576 tablet_bootstrap.cc:654] T e225de9e71de4760a6d8a9a97b373eaa P b2b690173e9947beb7fa84eea32bb595: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:11.272253 21577 tablet_bootstrap.cc:654] T e225de9e71de4760a6d8a9a97b373eaa P bad95cef338d4797986b7a943ba43784: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:11.272415 21575 log.cc:824] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:11.272665 21576 log.cc:824] T e225de9e71de4760a6d8a9a97b373eaa P b2b690173e9947beb7fa84eea32bb595: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:11.272871 21577 log.cc:824] T e225de9e71de4760a6d8a9a97b373eaa P bad95cef338d4797986b7a943ba43784: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:11.273329 21575 tablet_bootstrap.cc:492] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2: No bootstrap required, opened a new log
I20260812 16:32:11.273433 21575 ts_tablet_manager.cc:1397] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:32:11.273515 21576 tablet_bootstrap.cc:492] T e225de9e71de4760a6d8a9a97b373eaa P b2b690173e9947beb7fa84eea32bb595: No bootstrap required, opened a new log
I20260812 16:32:11.273571 21576 ts_tablet_manager.cc:1397] T e225de9e71de4760a6d8a9a97b373eaa P b2b690173e9947beb7fa84eea32bb595: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 16:32:11.273820 21577 tablet_bootstrap.cc:492] T e225de9e71de4760a6d8a9a97b373eaa P bad95cef338d4797986b7a943ba43784: No bootstrap required, opened a new log
I20260812 16:32:11.273900 21577 ts_tablet_manager.cc:1397] T e225de9e71de4760a6d8a9a97b373eaa P bad95cef338d4797986b7a943ba43784: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 16:32:11.275389 21576 raft_consensus.cc:348] T e225de9e71de4760a6d8a9a97b373eaa P b2b690173e9947beb7fa84eea32bb595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad95cef338d4797986b7a943ba43784" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 38939 } } peers { permanent_uuid: "fbbe5b2252174173af0a60365055bef2" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46173 } } peers { permanent_uuid: "b2b690173e9947beb7fa84eea32bb595" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 42653 } }
I20260812 16:32:11.275566 21576 raft_consensus.cc:374] T e225de9e71de4760a6d8a9a97b373eaa P b2b690173e9947beb7fa84eea32bb595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:11.275615 21576 raft_consensus.cc:729] T e225de9e71de4760a6d8a9a97b373eaa P b2b690173e9947beb7fa84eea32bb595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2b690173e9947beb7fa84eea32bb595, State: Initialized, Role: FOLLOWER
I20260812 16:32:11.275739 21576 consensus_queue.cc:260] T e225de9e71de4760a6d8a9a97b373eaa P b2b690173e9947beb7fa84eea32bb595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad95cef338d4797986b7a943ba43784" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 38939 } } peers { permanent_uuid: "fbbe5b2252174173af0a60365055bef2" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46173 } } peers { permanent_uuid: "b2b690173e9947beb7fa84eea32bb595" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 42653 } }
I20260812 16:32:11.275691 21575 raft_consensus.cc:348] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad95cef338d4797986b7a943ba43784" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 38939 } } peers { permanent_uuid: "fbbe5b2252174173af0a60365055bef2" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46173 } } peers { permanent_uuid: "b2b690173e9947beb7fa84eea32bb595" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 42653 } }
I20260812 16:32:11.275916 21575 raft_consensus.cc:374] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:11.275974 21575 raft_consensus.cc:729] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbbe5b2252174173af0a60365055bef2, State: Initialized, Role: FOLLOWER
I20260812 16:32:11.276073 21576 ts_tablet_manager.cc:1428] T e225de9e71de4760a6d8a9a97b373eaa P b2b690173e9947beb7fa84eea32bb595: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:32:11.276129 21253 heartbeater.cc:503] Master 127.19.238.126:37039 was elected leader, sending a full tablet report...
I20260812 16:32:11.276099 21577 raft_consensus.cc:348] T e225de9e71de4760a6d8a9a97b373eaa P bad95cef338d4797986b7a943ba43784 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad95cef338d4797986b7a943ba43784" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 38939 } } peers { permanent_uuid: "fbbe5b2252174173af0a60365055bef2" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46173 } } peers { permanent_uuid: "b2b690173e9947beb7fa84eea32bb595" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 42653 } }
I20260812 16:32:11.276257 21577 raft_consensus.cc:374] T e225de9e71de4760a6d8a9a97b373eaa P bad95cef338d4797986b7a943ba43784 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:11.276288 21577 raft_consensus.cc:729] T e225de9e71de4760a6d8a9a97b373eaa P bad95cef338d4797986b7a943ba43784 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad95cef338d4797986b7a943ba43784, State: Initialized, Role: FOLLOWER
I20260812 16:32:11.276347 21575 consensus_queue.cc:260] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad95cef338d4797986b7a943ba43784" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 38939 } } peers { permanent_uuid: "fbbe5b2252174173af0a60365055bef2" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46173 } } peers { permanent_uuid: "b2b690173e9947beb7fa84eea32bb595" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 42653 } }
I20260812 16:32:11.276423 21577 consensus_queue.cc:260] T e225de9e71de4760a6d8a9a97b373eaa P bad95cef338d4797986b7a943ba43784 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad95cef338d4797986b7a943ba43784" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 38939 } } peers { permanent_uuid: "fbbe5b2252174173af0a60365055bef2" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46173 } } peers { permanent_uuid: "b2b690173e9947beb7fa84eea32bb595" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 42653 } }
I20260812 16:32:11.276679 21575 ts_tablet_manager.cc:1428] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:32:11.276721 21577 ts_tablet_manager.cc:1428] T e225de9e71de4760a6d8a9a97b373eaa P bad95cef338d4797986b7a943ba43784: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 16:32:11.276731 21555 heartbeater.cc:503] Master 127.19.238.126:37039 was elected leader, sending a full tablet report...
I20260812 16:32:11.276870 21403 heartbeater.cc:503] Master 127.19.238.126:37039 was elected leader, sending a full tablet report...
I20260812 16:32:11.283404 21583 raft_consensus.cc:482] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:32:11.283492 21583 raft_consensus.cc:504] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad95cef338d4797986b7a943ba43784" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 38939 } } peers { permanent_uuid: "fbbe5b2252174173af0a60365055bef2" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46173 } } peers { permanent_uuid: "b2b690173e9947beb7fa84eea32bb595" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 42653 } }
I20260812 16:32:11.283811 21583 leader_election.cc:288] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad95cef338d4797986b7a943ba43784 (127.19.238.67:38939), b2b690173e9947beb7fa84eea32bb595 (127.19.238.65:42653)
I20260812 16:32:11.286772 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:11.284006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:42653 (local address 127.19.238.66:36909)
0812 16:32:11.284086 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:11.284115 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:11.284161 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:11.284336 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:11.284337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:11.284339 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:11.284422 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:11.284425 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.285140 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.285142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:11.285930 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:11.285932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.286490 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.286491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:11.286577 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:11.286580 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:32:11.286661 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 16:32:11.286700 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 16:32:11.286772 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:11.284080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:36909 (local address 127.19.238.65:42653)
0812 16:32:11.284151 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:11.284153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:11.284163 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:11.284241 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:11.284242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:11.284252 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:11.284303 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:11.284508 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.285050 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.286024 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.286410 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.286493 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:11.286502 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:11.286671 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 16:32:11.286707 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 16:32:11.287191 21200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e225de9e71de4760a6d8a9a97b373eaa" candidate_uuid: "fbbe5b2252174173af0a60365055bef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2b690173e9947beb7fa84eea32bb595" is_pre_election: true
I20260812 16:32:11.287274 21572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:11.284216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:51411 (local address 127.19.238.67:38939)
0812 16:32:11.284266 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:11.284268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:11.284278 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:11.284343 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:11.284344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:11.284355 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:11.284387 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:11.284673 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.285212 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.286295 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.286708 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.286812 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:11.286821 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:11.287164 (+ 343us) server_negotiation.cc:299] Negotiation successful
0812 16:32:11.287206 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 16:32:11.287284 21585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:11.284011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:38939 (local address 127.19.238.66:51411)
0812 16:32:11.284193 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:11.284205 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:11.284267 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:11.284470 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:11.284471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:11.284473 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:11.284545 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:11.284546 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.285306 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.285308 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:11.286193 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:11.286195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.286992 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.286994 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:11.287077 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:11.287081 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:11.287158 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 16:32:11.287199 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":13,"thread_start_us":109,"threads_started":1}
I20260812 16:32:11.287360 21200 raft_consensus.cc:2454] T e225de9e71de4760a6d8a9a97b373eaa P b2b690173e9947beb7fa84eea32bb595 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbbe5b2252174173af0a60365055bef2 in term 0.
I20260812 16:32:11.287612 21281 leader_election.cc:302] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2b690173e9947beb7fa84eea32bb595, fbbe5b2252174173af0a60365055bef2; no voters:
I20260812 16:32:11.287806 21583 raft_consensus.cc:2781] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:32:11.287791 21504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e225de9e71de4760a6d8a9a97b373eaa" candidate_uuid: "fbbe5b2252174173af0a60365055bef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad95cef338d4797986b7a943ba43784" is_pre_election: true
I20260812 16:32:11.287930 21583 raft_consensus.cc:482] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:32:11.287983 21583 raft_consensus.cc:3037] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:11.287988 21504 raft_consensus.cc:2454] T e225de9e71de4760a6d8a9a97b373eaa P bad95cef338d4797986b7a943ba43784 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbbe5b2252174173af0a60365055bef2 in term 0.
I20260812 16:32:11.288816 21583 raft_consensus.cc:504] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad95cef338d4797986b7a943ba43784" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 38939 } } peers { permanent_uuid: "fbbe5b2252174173af0a60365055bef2" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46173 } } peers { permanent_uuid: "b2b690173e9947beb7fa84eea32bb595" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 42653 } }
I20260812 16:32:11.289036 21583 leader_election.cc:288] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2 [CANDIDATE]: Term 1 election: Requested vote from peers bad95cef338d4797986b7a943ba43784 (127.19.238.67:38939), b2b690173e9947beb7fa84eea32bb595 (127.19.238.65:42653)
I20260812 16:32:11.289141 21200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e225de9e71de4760a6d8a9a97b373eaa" candidate_uuid: "fbbe5b2252174173af0a60365055bef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2b690173e9947beb7fa84eea32bb595"
I20260812 16:32:11.289230 21200 raft_consensus.cc:3037] T e225de9e71de4760a6d8a9a97b373eaa P b2b690173e9947beb7fa84eea32bb595 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:11.289984 21200 raft_consensus.cc:2454] T e225de9e71de4760a6d8a9a97b373eaa P b2b690173e9947beb7fa84eea32bb595 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbbe5b2252174173af0a60365055bef2 in term 1.
I20260812 16:32:11.290127 21504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e225de9e71de4760a6d8a9a97b373eaa" candidate_uuid: "fbbe5b2252174173af0a60365055bef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad95cef338d4797986b7a943ba43784"
I20260812 16:32:11.290150 21281 leader_election.cc:302] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2b690173e9947beb7fa84eea32bb595, fbbe5b2252174173af0a60365055bef2; no voters:
I20260812 16:32:11.290210 21504 raft_consensus.cc:3037] T e225de9e71de4760a6d8a9a97b373eaa P bad95cef338d4797986b7a943ba43784 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:11.290269 21583 raft_consensus.cc:2781] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:11.290500 21583 raft_consensus.cc:686] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2 [term 1 LEADER]: Becoming Leader. State: Replica: fbbe5b2252174173af0a60365055bef2, State: Running, Role: LEADER
I20260812 16:32:11.290607 21583 consensus_queue.cc:237] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad95cef338d4797986b7a943ba43784" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 38939 } } peers { permanent_uuid: "fbbe5b2252174173af0a60365055bef2" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46173 } } peers { permanent_uuid: "b2b690173e9947beb7fa84eea32bb595" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 42653 } }
I20260812 16:32:11.291199 21504 raft_consensus.cc:2454] T e225de9e71de4760a6d8a9a97b373eaa P bad95cef338d4797986b7a943ba43784 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbbe5b2252174173af0a60365055bef2 in term 1.
I20260812 16:32:11.291666 21041 catalog_manager.cc:5495] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbbe5b2252174173af0a60365055bef2 (127.19.238.66). New cstate: current_term: 1 leader_uuid: "fbbe5b2252174173af0a60365055bef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad95cef338d4797986b7a943ba43784" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 38939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbbe5b2252174173af0a60365055bef2" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2b690173e9947beb7fa84eea32bb595" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 42653 } health_report { overall_health: UNKNOWN } } }
Aug 12 16:32:11 dist-test-slave-mvvj krb5kdc[20995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.65@KRBTEST.COM
I20260812 16:32:11.308760 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:11.300239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47738 (local address 127.19.238.65:42653)
0812 16:32:11.300332 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:11.300335 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:11.300418 (+ 83us) server_negotiation.cc:407] Connection header received
0812 16:32:11.300522 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:11.300523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:11.300535 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:11.300601 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:11.301277 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.301821 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.302644 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.302881 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.302981 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:11.305758 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:11.305772 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:11.305772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:11.305780 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:11.307466 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:11.307826 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:11.307885 (+ 59us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:11.307886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:11.307957 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:11.308164 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:11.308164 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:11.308165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:11.308314 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:11.308490 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:11.308615 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 16:32:11.308687 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 16:32:11 dist-test-slave-mvvj krb5kdc[20995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552331, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260812 16:32:11.325695 21404 tablet.cc:2369] T e225de9e71de4760a6d8a9a97b373eaa P fbbe5b2252174173af0a60365055bef2: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 16:32:11 dist-test-slave-mvvj krb5kdc[20995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552331, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.238.65@KRBTEST.COM
I20260812 16:32:11.334918 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:11.326884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47752 (local address 127.19.238.65:42653)
0812 16:32:11.326969 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:11.326972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:11.327073 (+ 101us) server_negotiation.cc:407] Connection header received
0812 16:32:11.327183 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:11.327184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:11.327195 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:11.327276 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:11.328048 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.328601 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.329416 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.329687 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.329774 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:11.332474 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:11.332475 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:11.332476 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:11.332483 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:11.333839 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:11.334187 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:11.334188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:11.334188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:11.334241 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:11.334423 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:11.334424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:11.334424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:11.334526 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:11.334606 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:11.334764 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 16:32:11.334831 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 16:32:11 dist-test-slave-mvvj krb5kdc[20995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 16:32:11 dist-test-slave-mvvj krb5kdc[20995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.65@KRBTEST.COM
I20260812 16:32:11.360630 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:11.352684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47760 (local address 127.19.238.65:42653)
0812 16:32:11.352774 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:11.352777 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:11.352873 (+ 96us) server_negotiation.cc:407] Connection header received
0812 16:32:11.352993 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:11.352994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:11.353005 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:11.353082 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:11.353682 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.354174 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.354983 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:11.355177 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:11.355254 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:11.357655 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:11.357657 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:11.357657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:11.357665 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:11.359020 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:11.359451 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:11.359452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:11.359452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:11.359556 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:11.359762 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:11.359763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:11.359763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:11.360003 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:11.360097 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:11.360392 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 16:32:11.360468 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 16:32:11.361275 21157 tablet_service.cc:2908] Rejecting scan request for tablet e225de9e71de4760a6d8a9a97b373eaa: Uninitialized: clean time has not yet been initialized
I20260812 16:32:11.362237 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 21112
I20260812 16:32:11.370393 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 21260
I20260812 16:32:11.384088 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 21408
I20260812 16:32:11.392887 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 21008
2026-08-12T16:32:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2074 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:32:11 dist-test-slave-mvvj krb5kdc[21601](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:32:11 dist-test-slave-mvvj krb5kdc[21601](info): set up 2 sockets
Aug 12 16:32:11 dist-test-slave-mvvj krb5kdc[21601](info): commencing operation
krb5kdc: starting...
W20260812 16:32:12.524205 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:32:12 dist-test-slave-mvvj krb5kdc[21601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:32:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:32:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:32:12.664098 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:38907
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:38907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:12.749843 21616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:12.750110 21616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:12.750149 21616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:12.751330 21616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:12.751408 21616 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 16:32:12.751430 21616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:12.751453 21616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:12.751482 21616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:12.751502 21616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:32:12.753152 21616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:38907
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:38907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:12.753458 21616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:12.753718 21616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:12.759052 21623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:12.759119 21622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:12.759146 21616 server_base.cc:1034] running on GCE node
W20260812 16:32:12.759119 21625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:12.759718 21616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:12.760131 21616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:12.761303 21616 hybrid_clock.cc:648] HybridClock initialized: now 1786552332761280 us; error 34 us; skew 500 ppm
Aug 12 16:32:12 dist-test-slave-mvvj krb5kdc[21601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552332, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:12.770984 21616 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:32:12.771510 21616 webserver.cc:460] Webserver started at http://127.19.238.126:38339/ using document root <none> and password file <none>
I20260812 16:32:12.771759 21616 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:12.771804 21616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:12.771996 21616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:12.773188 21616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "0b92cc799aa14dc393b4043378886752"
format_stamp: "Formatted at 2026-08-12 16:32:12 on dist-test-slave-mvvj"
server_key: "bc8dda92e098cee6c1061a5e610dcb01"
server_key_iv: "b5b2416d1f926ee224fbf60d6848ea7f"
server_key_version: "encryptionkey@0"
I20260812 16:32:12.773530 21616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "0b92cc799aa14dc393b4043378886752"
format_stamp: "Formatted at 2026-08-12 16:32:12 on dist-test-slave-mvvj"
server_key: "bc8dda92e098cee6c1061a5e610dcb01"
server_key_iv: "b5b2416d1f926ee224fbf60d6848ea7f"
server_key_version: "encryptionkey@0"
I20260812 16:32:12.775064 21616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:32:12.776011 21633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:12.776203 21616 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:12.776311 21616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "0b92cc799aa14dc393b4043378886752"
format_stamp: "Formatted at 2026-08-12 16:32:12 on dist-test-slave-mvvj"
server_key: "bc8dda92e098cee6c1061a5e610dcb01"
server_key_iv: "b5b2416d1f926ee224fbf60d6848ea7f"
server_key_version: "encryptionkey@0"
I20260812 16:32:12.776382 21616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:12.804111 21616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:12.807250 21616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:12.807396 21616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:12.812719 21616 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:38907
I20260812 16:32:12.812819 21704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:38907 every 8 connection(s)
I20260812 16:32:12.813215 21616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:32:12.814026 21705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:12.816675 21705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752: Bootstrap starting.
I20260812 16:32:12.817363 21705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:12.817672 21705 log.cc:824] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:12.818593 21705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752: No bootstrap required, opened a new log
I20260812 16:32:12.821017 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 21616
I20260812 16:32:12.821121 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:32:12.821336 20409 external_mini_cluster.cc:1428] Setting key 96a7f0b8cab2e4cceb2c30744b27e12b
I20260812 16:32:12.821312 21705 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b92cc799aa14dc393b4043378886752" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38907 } }
I20260812 16:32:12.821547 21705 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:12.821622 21705 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b92cc799aa14dc393b4043378886752, State: Initialized, Role: FOLLOWER
I20260812 16:32:12.821820 21705 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b92cc799aa14dc393b4043378886752" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38907 } }
I20260812 16:32:12.821946 21705 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:32:12.821983 21705 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:32:12.822031 21705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:12.822742 21705 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b92cc799aa14dc393b4043378886752" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38907 } }
I20260812 16:32:12.822885 21705 leader_election.cc:302] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b92cc799aa14dc393b4043378886752; no voters:
I20260812 16:32:12.823122 21705 leader_election.cc:288] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:32:12.823149 21710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:12.823282 21710 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752 [term 1 LEADER]: Becoming Leader. State: Replica: 0b92cc799aa14dc393b4043378886752, State: Running, Role: LEADER
I20260812 16:32:12.823424 21705 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:12.823452 21710 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b92cc799aa14dc393b4043378886752" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38907 } }
I20260812 16:32:12.823804 21710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b92cc799aa14dc393b4043378886752. Latest consensus state: current_term: 1 leader_uuid: "0b92cc799aa14dc393b4043378886752" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b92cc799aa14dc393b4043378886752" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38907 } } }
I20260812 16:32:12.823913 21710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:12.824143 21715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:32:12.824717 21715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:32:12.825821 21712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b92cc799aa14dc393b4043378886752" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b92cc799aa14dc393b4043378886752" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38907 } } }
I20260812 16:32:12.825922 21712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:12.827059 21715 catalog_manager.cc:1347] Generated new cluster ID: cc146acd4e574fff8782054fae63751d
I20260812 16:32:12.827109 21715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:32:12 dist-test-slave-mvvj krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:12.834622 21709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:12.822022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41686 (local address 127.19.238.126:38907)
0812 16:32:12.822265 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:12.822276 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:12.822318 (+ 42us) server_negotiation.cc:407] Connection header received
0812 16:32:12.822441 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:12.822447 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:12.822498 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:12.822657 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:12.823574 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:12.825245 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:12.826417 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:12.827029 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:12.827155 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:12.829927 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:12.829931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:12.829931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:12.829946 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:12.832712 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:12.833184 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:12.833188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:12.833189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:12.833319 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:12.833570 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:12.833571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:12.833572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:12.833867 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:12.834077 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:12.834187 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 16:32:12.834363 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":119,"threads_started":1}
I20260812 16:32:12.853164 21715 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:32:12.853848 21715 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:12.865624 21715 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752: Generated new TSK 0
I20260812 16:32:12.865877 21715 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
I20260812 16:32:12.903666 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:38907
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:13.001361 21734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:13.001601 21734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:13.001631 21734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:13.002816 21734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:13.002868 21734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:13.002889 21734 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:32:13.002908 21734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:32:13.004415 21734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.238.126:38907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:13.004634 21734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:13.004837 21734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:13.010419 21743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:13.010485 21741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:13.010506 21740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:13.010641 21734 server_base.cc:1034] running on GCE node
I20260812 16:32:13.010943 21734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:13.011234 21734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:13.012431 21734 hybrid_clock.cc:648] HybridClock initialized: now 1786552333012421 us; error 56 us; skew 500 ppm
Aug 12 16:32:13 dist-test-slave-mvvj krb5kdc[21601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552333, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:13.016119 21734 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:32:13.016674 21734 webserver.cc:460] Webserver started at http://127.19.238.65:45729/ using document root <none> and password file <none>
I20260812 16:32:13.016945 21734 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:13.017020 21734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:13.017164 21734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:13.018313 21734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "727655effcd8424f8eff83ad83d111b7"
format_stamp: "Formatted at 2026-08-12 16:32:13 on dist-test-slave-mvvj"
server_key: "becba4a1d0b8b76b4dc2b3be9cbddff3"
server_key_iv: "d9c93829d56fad88485452cbaa2da7eb"
server_key_version: "encryptionkey@0"
I20260812 16:32:13.018719 21734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "727655effcd8424f8eff83ad83d111b7"
format_stamp: "Formatted at 2026-08-12 16:32:13 on dist-test-slave-mvvj"
server_key: "becba4a1d0b8b76b4dc2b3be9cbddff3"
server_key_iv: "d9c93829d56fad88485452cbaa2da7eb"
server_key_version: "encryptionkey@0"
I20260812 16:32:13.020254 21734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:32:13.021306 21750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:13.021580 21734 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:13.021696 21734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "727655effcd8424f8eff83ad83d111b7"
format_stamp: "Formatted at 2026-08-12 16:32:13 on dist-test-slave-mvvj"
server_key: "becba4a1d0b8b76b4dc2b3be9cbddff3"
server_key_iv: "d9c93829d56fad88485452cbaa2da7eb"
server_key_version: "encryptionkey@0"
I20260812 16:32:13.021806 21734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:13.033879 21734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:13.036353 21734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:13.036475 21734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:13.037002 21734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:13.037067 21734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:13.037111 21734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:13.037142 21734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:13.044709 21734 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:35105
I20260812 16:32:13.044800 21875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:35105 every 8 connection(s)
Aug 12 16:32:13 dist-test-slave-mvvj krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552333, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:13.045184 21734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:32:13.048910 21709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.037819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:55563 (local address 127.19.238.126:38907)
0812 16:32:13.037877 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:13.037879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:13.038156 (+ 277us) server_negotiation.cc:407] Connection header received
0812 16:32:13.038902 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:13.038904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:13.038925 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:13.039007 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:13.040080 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.040973 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.042204 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.042600 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.042709 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.045603 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:13.045605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:13.045606 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:13.045618 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:13.047246 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:13.047738 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:13.047740 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:13.047740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:13.047808 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:13.048184 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:13.048186 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:13.048186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:13.048319 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:13.048395 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:13.048748 (+ 353us) server_negotiation.cc:299] Negotiation successful
0812 16:32:13.048842 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 16:32:13.048993 21759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.037808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:38907 (local address 127.19.238.65:55563)
0812 16:32:13.038012 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:13.038052 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:13.038747 (+ 695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:13.039117 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:13.039123 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:13.039579 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:13.039945 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.039952 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.041104 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.041105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.042092 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.042095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.042723 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.042724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.042887 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.043557 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:13.043576 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:13.045463 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:13.047382 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:13.047390 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:13.047391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:13.047631 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:13.047960 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:13.047961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:13.047961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:13.048083 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:13.048435 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:13.048441 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:13.048577 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 16:32:13.048682 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 16:32:13.048819 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":97,"threads_started":1}
I20260812 16:32:13.049754 21878 heartbeater.cc:347] Connected to a master server at 127.19.238.126:38907
I20260812 16:32:13.049876 21878 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:13.050071 21878 heartbeater.cc:511] Master 127.19.238.126:38907 requested a full tablet report, sending...
I20260812 16:32:13.050765 21653 ts_manager.cc:194] Registered new tserver with Master: 727655effcd8424f8eff83ad83d111b7 (127.19.238.65:35105)
I20260812 16:32:13.051272 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 21734
I20260812 16:32:13.051381 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:32:13.051566 20409 external_mini_cluster.cc:1428] Setting key 94e18e8bfa929d4167e89994b697f5d9
I20260812 16:32:13.051579 21653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:55563
I20260812 16:32:13.057986 21709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.051044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:34107 (local address 127.19.238.126:38907)
0812 16:32:13.051101 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:13.051104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:13.051963 (+ 859us) server_negotiation.cc:407] Connection header received
0812 16:32:13.052039 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:13.052040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:13.052052 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:13.052106 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:13.052579 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.053123 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.053836 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.054054 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.054147 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.055201 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:13.055203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:13.055203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:13.055211 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:13.056724 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:13.057111 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:13.057112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:13.057112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:13.057169 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:13.057398 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:13.057399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:13.057400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:13.057544 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:13.057620 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:13.057790 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 16:32:13.057892 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 16:32:13.057999 21884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.051002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:38907 (local address 127.19.238.65:34107)
0812 16:32:13.051875 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:13.051887 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:13.051978 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:13.052155 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:13.052156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:13.052387 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:13.052461 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.052464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.053231 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.053232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.053742 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.053744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.054149 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.054151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.054242 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.054768 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:13.054773 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:13.055098 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:13.056839 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:13.056840 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:13.056841 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:13.057029 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:13.057252 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:13.057253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:13.057253 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:13.057318 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:13.057624 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:13.057625 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:13.057676 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 16:32:13.057762 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 16:32:13.057881 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for kudu/127.19.238.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
I20260812 16:32:13.082885 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:38907
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:13.167124 21887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:13.167333 21887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:13.167357 21887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:13.168579 21887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:13.168624 21887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:13.168645 21887 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:32:13.168663 21887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:32:13.170099 21887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.238.126:38907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:13.170358 21887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:13.170604 21887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:13.175796 21895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:13.175791 21887 server_base.cc:1034] running on GCE node
W20260812 16:32:13.175792 21893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:13.175859 21892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:13.176328 21887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:13.176631 21887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:13.177809 21887 hybrid_clock.cc:648] HybridClock initialized: now 1786552333177798 us; error 50 us; skew 500 ppm
Aug 12 16:32:13 dist-test-slave-mvvj krb5kdc[21601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552333, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:13.180663 21887 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:32:13.181263 21887 webserver.cc:460] Webserver started at http://127.19.238.66:36485/ using document root <none> and password file <none>
I20260812 16:32:13.181526 21887 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:13.181614 21887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:13.181775 21887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:13.182914 21887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "0f97c49dd136430e8641ca1376b86e09"
format_stamp: "Formatted at 2026-08-12 16:32:13 on dist-test-slave-mvvj"
server_key: "561b1e20245530db25ab7735635b81e8"
server_key_iv: "477dc5c16e4fea3ae58dbd4d74fe5445"
server_key_version: "encryptionkey@0"
I20260812 16:32:13.183295 21887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "0f97c49dd136430e8641ca1376b86e09"
format_stamp: "Formatted at 2026-08-12 16:32:13 on dist-test-slave-mvvj"
server_key: "561b1e20245530db25ab7735635b81e8"
server_key_iv: "477dc5c16e4fea3ae58dbd4d74fe5445"
server_key_version: "encryptionkey@0"
I20260812 16:32:13.184847 21887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:32:13.185812 21903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:13.186080 21887 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:13.186200 21887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "0f97c49dd136430e8641ca1376b86e09"
format_stamp: "Formatted at 2026-08-12 16:32:13 on dist-test-slave-mvvj"
server_key: "561b1e20245530db25ab7735635b81e8"
server_key_iv: "477dc5c16e4fea3ae58dbd4d74fe5445"
server_key_version: "encryptionkey@0"
I20260812 16:32:13.186283 21887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:13.196630 21887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:13.199772 21887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:13.199990 21887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:13.200565 21887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:13.200623 21887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:13.200655 21887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:13.200712 21887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:13.208272 21887 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:43977
Aug 12 16:32:13 dist-test-slave-mvvj krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552333, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:13.208837 21887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:32:13.209038 22035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:43977 every 8 connection(s)
I20260812 16:32:13.212810 21709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.201160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:36277 (local address 127.19.238.126:38907)
0812 16:32:13.201214 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:13.201216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:13.201470 (+ 254us) server_negotiation.cc:407] Connection header received
0812 16:32:13.202047 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:13.202049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:13.202062 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:13.202116 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:13.203074 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.203937 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.205971 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.206200 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.206264 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.209686 (+ 3422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:13.209688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:13.209688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:13.209697 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:13.211251 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:13.211754 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:13.211755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:13.211755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:13.211812 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:13.212160 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:13.212161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:13.212161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:13.212284 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:13.212341 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:13.212665 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 16:32:13.212739 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 16:32:13.213035 21914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.201165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:38907 (local address 127.19.238.66:36277)
0812 16:32:13.201352 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:13.201393 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:13.201937 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:13.202215 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:13.202220 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:13.202625 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:13.202925 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.202935 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.204062 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.204064 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.205823 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.205829 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.206315 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.206317 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.206523 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.207221 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:13.207235 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:13.209539 (+ 2304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:13.211347 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:13.211355 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:13.211357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:13.211675 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:13.211953 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:13.211954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:13.211955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:13.212088 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:13.212347 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:13.212352 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:13.212536 (+ 184us) client_negotiation.cc:769] Sending connection context
0812 16:32:13.212644 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 16:32:13.212786 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":111,"threads_started":1}
I20260812 16:32:13.213601 22036 heartbeater.cc:347] Connected to a master server at 127.19.238.126:38907
I20260812 16:32:13.213693 22036 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:13.213907 22036 heartbeater.cc:511] Master 127.19.238.126:38907 requested a full tablet report, sending...
I20260812 16:32:13.214365 21652 ts_manager.cc:194] Registered new tserver with Master: 0f97c49dd136430e8641ca1376b86e09 (127.19.238.66:43977)
I20260812 16:32:13.214918 21652 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:36277
I20260812 16:32:13.218874 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 21887
I20260812 16:32:13.218990 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:32:13.219213 20409 external_mini_cluster.cc:1428] Setting key 7c31340a0e7f1af10f815d1f4971abc2
I20260812 16:32:13.221015 22042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.214814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:38907 (local address 127.19.238.66:38913)
0812 16:32:13.215002 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:13.215016 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:13.215098 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:13.215232 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:13.215233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:13.215472 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:13.215537 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.215541 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.216270 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.216271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.216773 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.216775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.217174 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.217175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.217248 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.217796 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:13.217801 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:13.218128 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:13.219876 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:13.219877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:13.219877 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:13.220119 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:13.220336 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:13.220336 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:13.220337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:13.220394 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:13.220752 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:13.220753 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:13.220794 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 16:32:13.220851 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 16:32:13.220936 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":111,"threads_started":1}
I20260812 16:32:13.221149 21709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.214927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:38913 (local address 127.19.238.126:38907)
0812 16:32:13.214972 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:13.214974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:13.215080 (+ 106us) server_negotiation.cc:407] Connection header received
0812 16:32:13.215151 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:13.215152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:13.215163 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:13.215229 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:13.215623 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.216173 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.216857 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.217093 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.217155 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.218204 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:13.218206 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:13.218206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:13.218212 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:13.219752 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:13.220195 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:13.220197 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:13.220197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:13.220264 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:13.220458 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:13.220459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:13.220460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:13.220675 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:13.220752 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:13.220922 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 16:32:13.221013 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.19.238.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
I20260812 16:32:13.247213 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:38907
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:13.333498 22045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:13.333740 22045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:13.333787 22045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:13.335215 22045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:13.335285 22045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:13.335319 22045 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:32:13.335359 22045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:32:13.336995 22045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.238.126:38907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:13.337297 22045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:13.337558 22045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:13.343312 22051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:13.343349 22045 server_base.cc:1034] running on GCE node
W20260812 16:32:13.343359 22050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:13.343372 22053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:13.343888 22045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:13.344239 22045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:13.345474 22045 hybrid_clock.cc:648] HybridClock initialized: now 1786552333345456 us; error 62 us; skew 500 ppm
Aug 12 16:32:13 dist-test-slave-mvvj krb5kdc[21601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552333, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:13.348546 22045 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:32:13.349002 22045 webserver.cc:460] Webserver started at http://127.19.238.67:37453/ using document root <none> and password file <none>
I20260812 16:32:13.349351 22045 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:13.349416 22045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:13.349599 22045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:13.350872 22045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "3b65cb0ba707424c82542daf9dabe021"
format_stamp: "Formatted at 2026-08-12 16:32:13 on dist-test-slave-mvvj"
server_key: "c62302dbf75837383ee08f571ce83ebc"
server_key_iv: "7b9040fc00e958f08462f938d043211d"
server_key_version: "encryptionkey@0"
I20260812 16:32:13.351274 22045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "3b65cb0ba707424c82542daf9dabe021"
format_stamp: "Formatted at 2026-08-12 16:32:13 on dist-test-slave-mvvj"
server_key: "c62302dbf75837383ee08f571ce83ebc"
server_key_iv: "7b9040fc00e958f08462f938d043211d"
server_key_version: "encryptionkey@0"
I20260812 16:32:13.352981 22045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 16:32:13.353817 22060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:13.354014 22045 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:13.354110 22045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "3b65cb0ba707424c82542daf9dabe021"
format_stamp: "Formatted at 2026-08-12 16:32:13 on dist-test-slave-mvvj"
server_key: "c62302dbf75837383ee08f571ce83ebc"
server_key_iv: "7b9040fc00e958f08462f938d043211d"
server_key_version: "encryptionkey@0"
I20260812 16:32:13.354197 22045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:13.364758 22045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:13.367350 22045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:13.367532 22045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:13.368206 22045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:13.368276 22045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:13.368348 22045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:13.368381 22045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:13.375226 22045 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:46093
I20260812 16:32:13.375314 22194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:46093 every 8 connection(s)
I20260812 16:32:13.375713 22045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
Aug 12 16:32:13 dist-test-slave-mvvj krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552333, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:13.379340 21709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.368750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:36009 (local address 127.19.238.126:38907)
0812 16:32:13.368794 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:13.368796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:13.369047 (+ 251us) server_negotiation.cc:407] Connection header received
0812 16:32:13.369613 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:13.369614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:13.369629 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:13.369688 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:13.370596 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.371416 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.373080 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.373414 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.373460 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.376497 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:13.376498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:13.376499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:13.376508 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:13.377840 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:13.378272 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:13.378274 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:13.378274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:13.378359 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:13.378642 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:13.378643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:13.378644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:13.378771 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:13.378847 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:13.379157 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 16:32:13.379248 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:32:13.379526 22072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.368749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:38907 (local address 127.19.238.67:36009)
0812 16:32:13.368918 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:13.368956 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:13.369498 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:13.369764 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:13.369768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:13.370144 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:13.370454 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.370461 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.371552 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.371553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.372955 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.372960 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.373543 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.373543 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.373644 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.374351 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:13.374363 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:13.376371 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:13.377950 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:13.377957 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:13.377959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:13.378190 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:13.378448 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:13.378449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:13.378449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:13.378566 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:13.378852 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:13.378859 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:13.379009 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 16:32:13.379120 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 16:32:13.379287 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":105,"threads_started":1}
I20260812 16:32:13.380077 22195 heartbeater.cc:347] Connected to a master server at 127.19.238.126:38907
I20260812 16:32:13.380226 22195 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:13.380465 22195 heartbeater.cc:511] Master 127.19.238.126:38907 requested a full tablet report, sending...
I20260812 16:32:13.380939 21652 ts_manager.cc:194] Registered new tserver with Master: 3b65cb0ba707424c82542daf9dabe021 (127.19.238.67:46093)
I20260812 16:32:13.381549 21652 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:36009
I20260812 16:32:13.383217 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 22045
I20260812 16:32:13.383306 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:32:13.383467 20409 external_mini_cluster.cc:1428] Setting key ec0928f1dd721d1214caa57d36c21496
I20260812 16:32:13.384488 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:32:13.390266 22201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.381132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:38907 (local address 127.19.238.67:54227)
0812 16:32:13.381281 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:13.381292 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:13.381347 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:13.381484 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:13.381484 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:13.381722 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:13.381798 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.381801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.382745 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.382746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.383499 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.383503 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.384104 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.384105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.384186 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.386490 (+ 2304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:13.386495 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:13.386793 (+ 298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:13.388871 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:13.388873 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:13.388873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:13.389120 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:13.389523 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:13.389524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:13.389524 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:13.389604 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:13.389910 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:13.389911 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:13.389971 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 16:32:13.390049 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 16:32:13.390175 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260812 16:32:13.390537 21709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.381213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:54227 (local address 127.19.238.126:38907)
0812 16:32:13.381276 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:13.381278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:13.381340 (+ 62us) server_negotiation.cc:407] Connection header received
0812 16:32:13.381395 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:13.381396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:13.381407 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:13.381482 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:13.381864 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.382641 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.383626 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.384020 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.384094 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.386896 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:13.386898 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:13.386899 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:13.386908 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:13.388775 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:13.389254 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:13.389256 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:13.389256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:13.389424 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:13.389690 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:13.389691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:13.389692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:13.389830 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:13.389892 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:13.390094 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 16:32:13.390190 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 16:32:13 dist-test-slave-mvvj krb5kdc[21601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552333, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 16:32:13 dist-test-slave-mvvj krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552333, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:13.409857 21709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.402528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41692 (local address 127.19.238.126:38907)
0812 16:32:13.402605 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:13.402608 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:13.402624 (+ 16us) server_negotiation.cc:407] Connection header received
0812 16:32:13.402652 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:13.402652 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:13.402663 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:13.402720 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:13.403341 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.403900 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.404658 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.404840 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.404895 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.407225 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:13.407226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:13.407227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:13.407234 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:13.408547 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:13.408867 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:13.408867 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:13.408868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:13.408932 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:13.409327 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:13.409328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:13.409328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:13.409458 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:13.409549 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:13.409710 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 16:32:13.409780 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 16:32:13.411729 21652 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:32:13.413429 21652 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:32:13.420854 22217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.416591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41536 (local address 127.19.238.67:46093)
0812 16:32:13.416925 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:13.416928 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:13.416939 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:32:13.417020 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:13.417022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:13.417035 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:13.417108 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:13.417324 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.418499 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.419759 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.420377 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.420468 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.420532 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:13.420692 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 16:32:13.420737 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":86,"threads_started":1}
I20260812 16:32:13.420874 22213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.416233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:43977 (local address 127.0.0.1:53628)
0812 16:32:13.416592 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:13.416601 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:13.416644 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:13.417216 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:13.417217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:13.417219 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:13.417258 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.417260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.418681 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.418683 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.419522 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.419525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.420556 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.420557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.420624 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.420628 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:13.420660 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 16:32:13.420756 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":112,"threads_started":1}
I20260812 16:32:13.420869 22214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.416236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:46093 (local address 127.0.0.1:41536)
0812 16:32:13.416860 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:13.416879 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:13.416933 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:13.417117 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:13.417118 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:13.417121 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:13.417225 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.417228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.418615 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.418616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.419669 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.419671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.420468 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.420469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.420552 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.420567 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 16:32:13.420672 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 16:32:13.420752 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":25,"thread_start_us":186,"threads_started":1}
I20260812 16:32:13.420886 22215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.416224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:35105 (local address 127.0.0.1:54004)
0812 16:32:13.416484 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:13.416526 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:13.416600 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:13.417113 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:13.417116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:13.417118 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:13.417194 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.417196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.418214 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.418216 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.419515 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.419518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.420615 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.420616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.420693 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.420696 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:32:13.420745 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 16:32:13.420784 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":1,"thread_start_us":67,"threads_started":1}
I20260812 16:32:13.421205 22219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.416660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53628 (local address 127.19.238.66:43977)
0812 16:32:13.416807 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:13.417008 (+ 201us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:13.417019 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:32:13.417092 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:13.417094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:13.417111 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:13.417198 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:13.417334 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.418546 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.419658 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.420450 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.420919 (+ 469us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.420994 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:13.421024 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 16:32:13.421076 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260812 16:32:13.421693 22216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.416428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54004 (local address 127.19.238.65:35105)
0812 16:32:13.416815 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:13.416820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:13.416831 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:32:13.416932 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:13.416935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:13.416954 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:13.417028 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:13.417302 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.418127 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.419618 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.420450 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.420526 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.420586 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:13.421255 (+ 669us) server_negotiation.cc:299] Negotiation successful
0812 16:32:13.421562 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":115,"threads_started":1}
I20260812 16:32:13.421993 21963 tablet_service.cc:1467] Processing CreateTablet for tablet ea30314504fd4653a4452f7bd4c96049 (DEFAULT_TABLE table=test-table [id=defb37d8af604df1b03f28cd7c5c9720]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:13.422379 21963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea30314504fd4653a4452f7bd4c96049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:13.422387 21805 tablet_service.cc:1467] Processing CreateTablet for tablet ea30314504fd4653a4452f7bd4c96049 (DEFAULT_TABLE table=test-table [id=defb37d8af604df1b03f28cd7c5c9720]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:13.422765 21805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea30314504fd4653a4452f7bd4c96049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:13.424307 22118 tablet_service.cc:1467] Processing CreateTablet for tablet ea30314504fd4653a4452f7bd4c96049 (DEFAULT_TABLE table=test-table [id=defb37d8af604df1b03f28cd7c5c9720]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:13.424734 22118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea30314504fd4653a4452f7bd4c96049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:13.426182 22220 tablet_bootstrap.cc:492] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7: Bootstrap starting.
I20260812 16:32:13.426676 22221 tablet_bootstrap.cc:492] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09: Bootstrap starting.
I20260812 16:32:13.426952 22220 tablet_bootstrap.cc:654] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:13.427300 22220 log.cc:824] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:13.427332 22221 tablet_bootstrap.cc:654] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:13.427665 22221 log.cc:824] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:13.428000 22224 tablet_bootstrap.cc:492] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021: Bootstrap starting.
I20260812 16:32:13.428304 22220 tablet_bootstrap.cc:492] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7: No bootstrap required, opened a new log
I20260812 16:32:13.428385 22220 ts_tablet_manager.cc:1397] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:32:13.428874 22224 tablet_bootstrap.cc:654] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:13.429178 22224 log.cc:824] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:13.429843 22224 tablet_bootstrap.cc:492] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021: No bootstrap required, opened a new log
I20260812 16:32:13.429895 22224 ts_tablet_manager.cc:1397] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:32:13.429904 22220 raft_consensus.cc:348] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.430091 22220 raft_consensus.cc:374] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:13.430140 22220 raft_consensus.cc:729] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 727655effcd8424f8eff83ad83d111b7, State: Initialized, Role: FOLLOWER
I20260812 16:32:13.430258 22220 consensus_queue.cc:260] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.430598 22220 ts_tablet_manager.cc:1428] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:32:13.430609 21878 heartbeater.cc:503] Master 127.19.238.126:38907 was elected leader, sending a full tablet report...
I20260812 16:32:13.431277 22221 tablet_bootstrap.cc:492] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09: No bootstrap required, opened a new log
I20260812 16:32:13.431365 22221 ts_tablet_manager.cc:1397] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260812 16:32:13.431790 22224 raft_consensus.cc:348] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.431990 22224 raft_consensus.cc:374] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:13.432027 22224 raft_consensus.cc:729] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b65cb0ba707424c82542daf9dabe021, State: Initialized, Role: FOLLOWER
I20260812 16:32:13.432160 22224 consensus_queue.cc:260] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.432420 22224 ts_tablet_manager.cc:1428] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 16:32:13.432466 22195 heartbeater.cc:503] Master 127.19.238.126:38907 was elected leader, sending a full tablet report...
I20260812 16:32:13.433655 22221 raft_consensus.cc:348] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.433780 22221 raft_consensus.cc:374] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:13.433801 22221 raft_consensus.cc:729] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f97c49dd136430e8641ca1376b86e09, State: Initialized, Role: FOLLOWER
I20260812 16:32:13.433911 22221 consensus_queue.cc:260] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.434149 22221 ts_tablet_manager.cc:1428] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 16:32:13.434263 22036 heartbeater.cc:503] Master 127.19.238.126:38907 was elected leader, sending a full tablet report...
W20260812 16:32:13.459547 22037 tablet.cc:2369] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:32:13.546020 21879 tablet.cc:2369] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:32:13.626464 22196 tablet.cc:2369] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:32:13.705222 22227 raft_consensus.cc:482] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:32:13.705556 22227 raft_consensus.cc:504] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.706028 22227 leader_election.cc:288] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f97c49dd136430e8641ca1376b86e09 (127.19.238.66:43977), 3b65cb0ba707424c82542daf9dabe021 (127.19.238.67:46093)
I20260812 16:32:13.710393 22219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.706359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:46939 (local address 127.19.238.66:43977)
0812 16:32:13.706476 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:13.706479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:13.706542 (+ 63us) server_negotiation.cc:407] Connection header received
0812 16:32:13.706610 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:13.706611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:13.706620 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:13.706675 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:13.706867 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.707646 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.709281 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.709926 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.710023 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.710033 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:13.710264 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 16:32:13.710314 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 16:32:13.710415 22217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.706440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:41893 (local address 127.19.238.67:46093)
0812 16:32:13.706496 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:13.706498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:13.706508 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:13.706566 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:13.706567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:13.706576 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:13.706648 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:13.706866 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.707620 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.709280 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.709931 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.710023 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.710033 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:13.710275 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 16:32:13.710325 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 16:32:13.710506 22231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.706263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:43977 (local address 127.19.238.65:46939)
0812 16:32:13.706482 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:13.706492 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:13.706551 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:13.706696 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:13.706697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:13.706698 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:13.706777 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.706779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.707741 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.707742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.709137 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.709141 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.710050 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.710051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.710137 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.710144 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 16:32:13.710250 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 16:32:13.710306 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":59,"thread_start_us":91,"threads_started":1}
I20260812 16:32:13.710529 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.706262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:46093 (local address 127.19.238.65:41893)
0812 16:32:13.706416 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:13.706428 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:13.706495 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:13.706694 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:13.706696 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:13.706698 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:13.706778 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.706779 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.707741 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.707742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.709137 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.709141 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.710096 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.710098 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.710178 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.710183 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:32:13.710265 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 16:32:13.710324 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":80,"thread_start_us":89,"threads_started":1}
I20260812 16:32:13.710922 22140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea30314504fd4653a4452f7bd4c96049" candidate_uuid: "727655effcd8424f8eff83ad83d111b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b65cb0ba707424c82542daf9dabe021" is_pre_election: true
I20260812 16:32:13.710959 21983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea30314504fd4653a4452f7bd4c96049" candidate_uuid: "727655effcd8424f8eff83ad83d111b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f97c49dd136430e8641ca1376b86e09" is_pre_election: true
I20260812 16:32:13.711133 22140 raft_consensus.cc:2454] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 727655effcd8424f8eff83ad83d111b7 in term 0.
I20260812 16:32:13.711133 21983 raft_consensus.cc:2454] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 727655effcd8424f8eff83ad83d111b7 in term 0.
I20260812 16:32:13.711400 21751 leader_election.cc:302] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f97c49dd136430e8641ca1376b86e09, 727655effcd8424f8eff83ad83d111b7; no voters:
I20260812 16:32:13.711581 22227 raft_consensus.cc:2781] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:32:13.711680 22227 raft_consensus.cc:482] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:32:13.711726 22227 raft_consensus.cc:3037] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:13.712575 22227 raft_consensus.cc:504] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.712759 22227 leader_election.cc:288] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [CANDIDATE]: Term 1 election: Requested vote from peers 0f97c49dd136430e8641ca1376b86e09 (127.19.238.66:43977), 3b65cb0ba707424c82542daf9dabe021 (127.19.238.67:46093)
I20260812 16:32:13.712975 21983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea30314504fd4653a4452f7bd4c96049" candidate_uuid: "727655effcd8424f8eff83ad83d111b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f97c49dd136430e8641ca1376b86e09"
I20260812 16:32:13.713079 21983 raft_consensus.cc:3037] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:13.713058 22140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea30314504fd4653a4452f7bd4c96049" candidate_uuid: "727655effcd8424f8eff83ad83d111b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b65cb0ba707424c82542daf9dabe021"
I20260812 16:32:13.713171 22140 raft_consensus.cc:3037] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:13.714001 21983 raft_consensus.cc:2454] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 727655effcd8424f8eff83ad83d111b7 in term 1.
I20260812 16:32:13.714058 22140 raft_consensus.cc:2454] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 727655effcd8424f8eff83ad83d111b7 in term 1.
I20260812 16:32:13.714210 21751 leader_election.cc:302] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f97c49dd136430e8641ca1376b86e09, 727655effcd8424f8eff83ad83d111b7; no voters:
I20260812 16:32:13.714326 22227 raft_consensus.cc:2781] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:13.714489 22227 raft_consensus.cc:686] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [term 1 LEADER]: Becoming Leader. State: Replica: 727655effcd8424f8eff83ad83d111b7, State: Running, Role: LEADER
I20260812 16:32:13.714670 22227 consensus_queue.cc:237] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.715646 21653 catalog_manager.cc:5495] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 727655effcd8424f8eff83ad83d111b7 (127.19.238.65). New cstate: current_term: 1 leader_uuid: "727655effcd8424f8eff83ad83d111b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } health_report { overall_health: UNKNOWN } } }
I20260812 16:32:13.780627 22216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.777562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54020 (local address 127.19.238.65:35105)
0812 16:32:13.777726 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:13.777728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:13.777739 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:32:13.777793 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:13.777795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:13.777814 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:13.777879 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:13.778037 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.778567 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.779594 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.779785 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.779865 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.780138 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:13.780250 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:13.780315 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:13.780495 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 16:32:13.780525 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260812 16:32:13.782609 21983 raft_consensus.cc:1261] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09 [term 1 FOLLOWER]: Refusing update from remote peer 727655effcd8424f8eff83ad83d111b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:13.782613 22140 raft_consensus.cc:1261] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021 [term 1 FOLLOWER]: Refusing update from remote peer 727655effcd8424f8eff83ad83d111b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:13.782977 22227 consensus_queue.cc:1035] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:13.783169 22232 consensus_queue.cc:1035] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:13.785727 22238 mvcc.cc:205] Tried to move back new op lower bound from 7317718359170580480 to 7317718358895972352. Current Snapshot: MvccSnapshot[applied={T|T < 7317718359170580480}]
I20260812 16:32:13.785768 22237 mvcc.cc:205] Tried to move back new op lower bound from 7317718359170580480 to 7317718358895972352. Current Snapshot: MvccSnapshot[applied={T|T < 7317718359170580480}]
I20260812 16:32:13.785478 22236 mvcc.cc:205] Tried to move back new op lower bound from 7317718359170580480 to 7317718358895972352. Current Snapshot: MvccSnapshot[applied={T|T < 7317718359170580480}]
I20260812 16:32:13.790661 21651 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:13.790814 21651 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:13.791858 21651 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752: Sending DeleteTablet for 3 replicas of tablet ea30314504fd4653a4452f7bd4c96049
I20260812 16:32:13.792264 21805 tablet_service.cc:1514] Processing DeleteTablet for tablet ea30314504fd4653a4452f7bd4c96049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:13 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:54004
I20260812 16:32:13.792244 21963 tablet_service.cc:1514] Processing DeleteTablet for tablet ea30314504fd4653a4452f7bd4c96049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:13 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:53628
I20260812 16:32:13.792510 22248 tablet_replica.cc:330] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7: stopping tablet replica
I20260812 16:32:13.792510 22247 tablet_replica.cc:330] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09: stopping tablet replica
I20260812 16:32:13.792526 22118 tablet_service.cc:1514] Processing DeleteTablet for tablet ea30314504fd4653a4452f7bd4c96049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:13 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:41536
I20260812 16:32:13.792618 22248 raft_consensus.cc:2229] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:32:13.792629 22247 raft_consensus.cc:2229] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:13.792734 22247 raft_consensus.cc:2258] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:13.792749 22248 raft_consensus.cc:2258] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:13.793265 22248 ts_tablet_manager.cc:1905] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:13.793354 21651 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:32:13.793522 21651 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:32:13.793835 22249 tablet_replica.cc:330] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021: stopping tablet replica
I20260812 16:32:13.794024 22249 raft_consensus.cc:2229] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:13.794216 22249 raft_consensus.cc:2258] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:13.795053 22118 tablet_service.cc:1467] Processing CreateTablet for tablet 4a586ae798654f0aac15d57262d2b1ca (DEFAULT_TABLE table=test-table [id=08a9eef9ce9547dea78b2fd3a7b6c473]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:13.795222 22118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a586ae798654f0aac15d57262d2b1ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:13.795248 22248 ts_tablet_manager.cc:1918] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:32:13.795323 22248 log.cc:1196] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-0/wal/wals/ea30314504fd4653a4452f7bd4c96049
I20260812 16:32:13.795223 22249 ts_tablet_manager.cc:1905] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:13.795624 22248 ts_tablet_manager.cc:1939] T ea30314504fd4653a4452f7bd4c96049 P 727655effcd8424f8eff83ad83d111b7: Deleting consensus metadata
I20260812 16:32:13.796734 22224 tablet_bootstrap.cc:492] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021: Bootstrap starting.
I20260812 16:32:13.796917 21635 catalog_manager.cc:4841] TS 727655effcd8424f8eff83ad83d111b7 (127.19.238.65:35105): tablet ea30314504fd4653a4452f7bd4c96049 (table test-table [id=defb37d8af604df1b03f28cd7c5c9720]) successfully deleted
I20260812 16:32:13.797226 22224 tablet_bootstrap.cc:654] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:13.794988 21963 tablet_service.cc:1467] Processing CreateTablet for tablet 4a586ae798654f0aac15d57262d2b1ca (DEFAULT_TABLE table=test-table [id=08a9eef9ce9547dea78b2fd3a7b6c473]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:13.797397 21963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a586ae798654f0aac15d57262d2b1ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:13.797712 21805 tablet_service.cc:1467] Processing CreateTablet for tablet 4a586ae798654f0aac15d57262d2b1ca (DEFAULT_TABLE table=test-table [id=08a9eef9ce9547dea78b2fd3a7b6c473]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:13.797835 21805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a586ae798654f0aac15d57262d2b1ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:13.797917 22249 ts_tablet_manager.cc:1918] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:32:13.798031 22249 log.cc:1196] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/ea30314504fd4653a4452f7bd4c96049
I20260812 16:32:13.798374 22249 ts_tablet_manager.cc:1939] T ea30314504fd4653a4452f7bd4c96049 P 3b65cb0ba707424c82542daf9dabe021: Deleting consensus metadata
I20260812 16:32:13.798888 22221 tablet_bootstrap.cc:492] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09: Bootstrap starting.
I20260812 16:32:13.793264 22247 ts_tablet_manager.cc:1905] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:13.799296 22220 tablet_bootstrap.cc:492] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7: Bootstrap starting.
I20260812 16:32:13.799332 22221 tablet_bootstrap.cc:654] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:13.799705 22220 tablet_bootstrap.cc:654] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:13.799710 22224 tablet_bootstrap.cc:492] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021: No bootstrap required, opened a new log
I20260812 16:32:13.799757 22224 ts_tablet_manager.cc:1397] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 16:32:13.799917 21636 catalog_manager.cc:4841] TS 3b65cb0ba707424c82542daf9dabe021 (127.19.238.67:46093): tablet ea30314504fd4653a4452f7bd4c96049 (table test-table [id=defb37d8af604df1b03f28cd7c5c9720]) successfully deleted
I20260812 16:32:13.799953 22224 raft_consensus.cc:348] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.800035 22224 raft_consensus.cc:374] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:13.800064 22224 raft_consensus.cc:729] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b65cb0ba707424c82542daf9dabe021, State: Initialized, Role: FOLLOWER
I20260812 16:32:13.800130 22224 consensus_queue.cc:260] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.800222 22224 ts_tablet_manager.cc:1428] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:13.800758 22221 tablet_bootstrap.cc:492] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09: No bootstrap required, opened a new log
I20260812 16:32:13.800814 22221 ts_tablet_manager.cc:1397] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 16:32:13.800985 22221 raft_consensus.cc:348] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.801059 22221 raft_consensus.cc:374] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:13.801083 22221 raft_consensus.cc:729] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f97c49dd136430e8641ca1376b86e09, State: Initialized, Role: FOLLOWER
I20260812 16:32:13.801128 22221 consensus_queue.cc:260] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.801491 22221 ts_tablet_manager.cc:1428] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:13.801501 22247 ts_tablet_manager.cc:1918] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:32:13.801712 22247 log.cc:1196] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/ea30314504fd4653a4452f7bd4c96049
I20260812 16:32:13.801851 22220 tablet_bootstrap.cc:492] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7: No bootstrap required, opened a new log
I20260812 16:32:13.801908 22220 ts_tablet_manager.cc:1397] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 16:32:13.802067 22220 raft_consensus.cc:348] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.802160 22220 raft_consensus.cc:374] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:13.802181 22220 raft_consensus.cc:729] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 727655effcd8424f8eff83ad83d111b7, State: Initialized, Role: FOLLOWER
I20260812 16:32:13.802225 22220 consensus_queue.cc:260] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.802320 22220 ts_tablet_manager.cc:1428] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:13.802567 22247 ts_tablet_manager.cc:1939] T ea30314504fd4653a4452f7bd4c96049 P 0f97c49dd136430e8641ca1376b86e09: Deleting consensus metadata
I20260812 16:32:13.803110 21634 catalog_manager.cc:4841] TS 0f97c49dd136430e8641ca1376b86e09 (127.19.238.66:43977): tablet ea30314504fd4653a4452f7bd4c96049 (table test-table [id=defb37d8af604df1b03f28cd7c5c9720]) successfully deleted
I20260812 16:32:13.901085 22232 raft_consensus.cc:482] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:32:13.901209 22232 raft_consensus.cc:504] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.901427 22232 leader_election.cc:288] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f97c49dd136430e8641ca1376b86e09 (127.19.238.66:43977), 3b65cb0ba707424c82542daf9dabe021 (127.19.238.67:46093)
I20260812 16:32:13.901736 21983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a586ae798654f0aac15d57262d2b1ca" candidate_uuid: "727655effcd8424f8eff83ad83d111b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f97c49dd136430e8641ca1376b86e09" is_pre_election: true
I20260812 16:32:13.901732 22140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a586ae798654f0aac15d57262d2b1ca" candidate_uuid: "727655effcd8424f8eff83ad83d111b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b65cb0ba707424c82542daf9dabe021" is_pre_election: true
I20260812 16:32:13.901860 22140 raft_consensus.cc:2454] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 727655effcd8424f8eff83ad83d111b7 in term 0.
I20260812 16:32:13.901860 21983 raft_consensus.cc:2454] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 727655effcd8424f8eff83ad83d111b7 in term 0.
I20260812 16:32:13.902067 21753 leader_election.cc:302] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b65cb0ba707424c82542daf9dabe021, 727655effcd8424f8eff83ad83d111b7; no voters:
I20260812 16:32:13.902197 22232 raft_consensus.cc:2781] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:32:13.902244 22232 raft_consensus.cc:482] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:32:13.902320 22232 raft_consensus.cc:3037] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:13.903105 22232 raft_consensus.cc:504] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.903288 22232 leader_election.cc:288] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [CANDIDATE]: Term 1 election: Requested vote from peers 0f97c49dd136430e8641ca1376b86e09 (127.19.238.66:43977), 3b65cb0ba707424c82542daf9dabe021 (127.19.238.67:46093)
I20260812 16:32:13.903482 21983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a586ae798654f0aac15d57262d2b1ca" candidate_uuid: "727655effcd8424f8eff83ad83d111b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f97c49dd136430e8641ca1376b86e09"
I20260812 16:32:13.903498 22140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a586ae798654f0aac15d57262d2b1ca" candidate_uuid: "727655effcd8424f8eff83ad83d111b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b65cb0ba707424c82542daf9dabe021"
I20260812 16:32:13.903586 22140 raft_consensus.cc:3037] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:13.903604 21983 raft_consensus.cc:3037] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:13.904337 21983 raft_consensus.cc:2454] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 727655effcd8424f8eff83ad83d111b7 in term 1.
I20260812 16:32:13.904503 22140 raft_consensus.cc:2454] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 727655effcd8424f8eff83ad83d111b7 in term 1.
I20260812 16:32:13.904544 21751 leader_election.cc:302] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f97c49dd136430e8641ca1376b86e09, 727655effcd8424f8eff83ad83d111b7; no voters:
I20260812 16:32:13.904641 22232 raft_consensus.cc:2781] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:13.904690 22232 raft_consensus.cc:686] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [term 1 LEADER]: Becoming Leader. State: Replica: 727655effcd8424f8eff83ad83d111b7, State: Running, Role: LEADER
I20260812 16:32:13.904739 22232 consensus_queue.cc:237] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.905493 21651 catalog_manager.cc:5495] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 727655effcd8424f8eff83ad83d111b7 (127.19.238.65). New cstate: current_term: 1 leader_uuid: "727655effcd8424f8eff83ad83d111b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } health_report { overall_health: UNKNOWN } } }
Aug 12 16:32:13 dist-test-slave-mvvj krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552332, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:13.945000 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.936671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:38907 (local address 127.0.0.1:41694)
0812 16:32:13.936855 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:13.936870 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:13.936955 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:13.937141 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:13.937142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:13.937484 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:13.937570 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.937573 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.938314 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.938316 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.938908 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:13.938911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.939289 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.939290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:13.939350 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.939999 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:13.940004 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:13.941547 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:13.943490 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:13.943500 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:13.943503 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:13.943787 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:13.944178 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:13.944179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:13.944179 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:13.944250 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:13.944692 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:13.944700 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:13.944749 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 16:32:13.944807 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 16:32:13.944917 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
I20260812 16:32:13.945077 22262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:13.936748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41694 (local address 127.19.238.126:38907)
0812 16:32:13.936923 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:13.936925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:13.936937 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:32:13.937041 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:13.937042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:13.937054 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:13.937137 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:13.937680 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.938205 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.939006 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:13.939200 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:13.939279 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:13.941665 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:13.941667 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:13.941667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:13.941675 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:13.943369 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:13.943983 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:13.943984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:13.943984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:13.944082 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:13.944416 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:13.944417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:13.944417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:13.944592 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:13.944696 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:13.944873 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 16:32:13.944979 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260812 16:32:13.947434 21651 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:41694:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 16:32:13.947742 21651 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:32:13.949540 21805 tablet_service.cc:1467] Processing CreateTablet for tablet 149787aca59d48d09d60b1152673f30a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aef0b96784424217bd205585149c669e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:32:13.949542 21963 tablet_service.cc:1467] Processing CreateTablet for tablet 149787aca59d48d09d60b1152673f30a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aef0b96784424217bd205585149c669e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:32:13.949582 22118 tablet_service.cc:1467] Processing CreateTablet for tablet 149787aca59d48d09d60b1152673f30a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aef0b96784424217bd205585149c669e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:32:13.949745 21963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 149787aca59d48d09d60b1152673f30a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:13.949753 22118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 149787aca59d48d09d60b1152673f30a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:13.949730 21805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 149787aca59d48d09d60b1152673f30a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:13.951401 22220 tablet_bootstrap.cc:492] T 149787aca59d48d09d60b1152673f30a P 727655effcd8424f8eff83ad83d111b7: Bootstrap starting.
I20260812 16:32:13.951411 22224 tablet_bootstrap.cc:492] T 149787aca59d48d09d60b1152673f30a P 3b65cb0ba707424c82542daf9dabe021: Bootstrap starting.
I20260812 16:32:13.951725 22220 tablet_bootstrap.cc:654] T 149787aca59d48d09d60b1152673f30a P 727655effcd8424f8eff83ad83d111b7: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:13.951826 22224 tablet_bootstrap.cc:654] T 149787aca59d48d09d60b1152673f30a P 3b65cb0ba707424c82542daf9dabe021: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:13.952301 22221 tablet_bootstrap.cc:492] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09: Bootstrap starting.
I20260812 16:32:13.952499 22220 tablet_bootstrap.cc:492] T 149787aca59d48d09d60b1152673f30a P 727655effcd8424f8eff83ad83d111b7: No bootstrap required, opened a new log
I20260812 16:32:13.952548 22220 ts_tablet_manager.cc:1397] T 149787aca59d48d09d60b1152673f30a P 727655effcd8424f8eff83ad83d111b7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:13.952651 22224 tablet_bootstrap.cc:492] T 149787aca59d48d09d60b1152673f30a P 3b65cb0ba707424c82542daf9dabe021: No bootstrap required, opened a new log
I20260812 16:32:13.952648 22220 raft_consensus.cc:348] T 149787aca59d48d09d60b1152673f30a P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.952698 22220 raft_consensus.cc:374] T 149787aca59d48d09d60b1152673f30a P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:13.952713 22220 raft_consensus.cc:729] T 149787aca59d48d09d60b1152673f30a P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 727655effcd8424f8eff83ad83d111b7, State: Initialized, Role: FOLLOWER
I20260812 16:32:13.952702 22224 ts_tablet_manager.cc:1397] T 149787aca59d48d09d60b1152673f30a P 3b65cb0ba707424c82542daf9dabe021: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:13.952729 22221 tablet_bootstrap.cc:654] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:13.952771 22220 consensus_queue.cc:260] T 149787aca59d48d09d60b1152673f30a P 727655effcd8424f8eff83ad83d111b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.952867 22220 ts_tablet_manager.cc:1428] T 149787aca59d48d09d60b1152673f30a P 727655effcd8424f8eff83ad83d111b7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:13.952859 22224 raft_consensus.cc:348] T 149787aca59d48d09d60b1152673f30a P 3b65cb0ba707424c82542daf9dabe021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.952942 22224 raft_consensus.cc:374] T 149787aca59d48d09d60b1152673f30a P 3b65cb0ba707424c82542daf9dabe021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:13.952968 22224 raft_consensus.cc:729] T 149787aca59d48d09d60b1152673f30a P 3b65cb0ba707424c82542daf9dabe021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b65cb0ba707424c82542daf9dabe021, State: Initialized, Role: FOLLOWER
I20260812 16:32:13.952884 22232 tablet_replica.cc:439] T 149787aca59d48d09d60b1152673f30a P 727655effcd8424f8eff83ad83d111b7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } } }
I20260812 16:32:13.953006 22232 tablet_replica.cc:442] T 149787aca59d48d09d60b1152673f30a P 727655effcd8424f8eff83ad83d111b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:13.953022 22224 consensus_queue.cc:260] T 149787aca59d48d09d60b1152673f30a P 3b65cb0ba707424c82542daf9dabe021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.953146 22224 ts_tablet_manager.cc:1428] T 149787aca59d48d09d60b1152673f30a P 3b65cb0ba707424c82542daf9dabe021: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:13.953289 22228 tablet_replica.cc:439] T 149787aca59d48d09d60b1152673f30a P 3b65cb0ba707424c82542daf9dabe021: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } } }
I20260812 16:32:13.953404 22228 tablet_replica.cc:442] T 149787aca59d48d09d60b1152673f30a P 3b65cb0ba707424c82542daf9dabe021: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:13.953591 22221 tablet_bootstrap.cc:492] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09: No bootstrap required, opened a new log
I20260812 16:32:13.953655 22221 ts_tablet_manager.cc:1397] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:13.953812 22221 raft_consensus.cc:348] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.953903 22221 raft_consensus.cc:374] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:13.953923 22221 raft_consensus.cc:729] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f97c49dd136430e8641ca1376b86e09, State: Initialized, Role: FOLLOWER
I20260812 16:32:13.953994 22221 consensus_queue.cc:260] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:13.954108 22221 ts_tablet_manager.cc:1428] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:13.954172 22229 tablet_replica.cc:439] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } } }
I20260812 16:32:13.954303 22229 tablet_replica.cc:442] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:14.032492 22229 raft_consensus.cc:482] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:32:14.032655 22229 raft_consensus.cc:504] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:14.033078 22229 leader_election.cc:288] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 727655effcd8424f8eff83ad83d111b7 (127.19.238.65:35105), 3b65cb0ba707424c82542daf9dabe021 (127.19.238.67:46093)
I20260812 16:32:14.036477 22216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:14.033368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:35525 (local address 127.19.238.65:35105)
0812 16:32:14.033432 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:14.033434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:14.033605 (+ 171us) server_negotiation.cc:407] Connection header received
0812 16:32:14.033830 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:14.033831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:14.033846 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:14.033913 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:14.034094 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.034641 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.035675 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.036097 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.036183 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:14.036192 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:14.036350 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 16:32:14.036398 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 16:32:14.036496 22267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:14.033291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:35105 (local address 127.19.238.66:35525)
0812 16:32:14.033506 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:14.033516 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:14.033624 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:14.033924 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:14.033925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:14.033926 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:14.033989 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:14.033991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.034729 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.034730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:14.035552 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:14.035556 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.036203 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.036204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:14.036267 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:14.036271 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:14.036316 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 16:32:14.036347 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":44,"thread_start_us":123,"threads_started":1}
I20260812 16:32:14.036528 22217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:14.033380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:48537 (local address 127.19.238.67:46093)
0812 16:32:14.033431 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:14.033434 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:14.033609 (+ 175us) server_negotiation.cc:407] Connection header received
0812 16:32:14.033651 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:14.033652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:14.033664 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:14.033726 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:14.034046 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.034533 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.035644 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.036095 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.036178 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:14.036188 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:14.036408 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 16:32:14.036453 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 16:32:14.036514 22266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:14.033283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:46093 (local address 127.19.238.66:48537)
0812 16:32:14.033493 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:14.033503 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:14.033585 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:14.033866 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:14.033867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:14.033870 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:14.033960 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:14.033963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.034645 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.034646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:14.035541 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:14.035544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.036211 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.036212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:14.036294 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:14.036321 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 16:32:14.036378 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 16:32:14.036417 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":147,"threads_started":1}
I20260812 16:32:14.036795 21828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "149787aca59d48d09d60b1152673f30a" candidate_uuid: "0f97c49dd136430e8641ca1376b86e09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "727655effcd8424f8eff83ad83d111b7" is_pre_election: true
I20260812 16:32:14.036958 21828 raft_consensus.cc:2454] T 149787aca59d48d09d60b1152673f30a P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f97c49dd136430e8641ca1376b86e09 in term 0.
I20260812 16:32:14.037055 22140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "149787aca59d48d09d60b1152673f30a" candidate_uuid: "0f97c49dd136430e8641ca1376b86e09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b65cb0ba707424c82542daf9dabe021" is_pre_election: true
I20260812 16:32:14.037173 22140 raft_consensus.cc:2454] T 149787aca59d48d09d60b1152673f30a P 3b65cb0ba707424c82542daf9dabe021 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f97c49dd136430e8641ca1376b86e09 in term 0.
I20260812 16:32:14.037258 21905 leader_election.cc:302] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f97c49dd136430e8641ca1376b86e09, 727655effcd8424f8eff83ad83d111b7; no voters:
I20260812 16:32:14.037406 22229 raft_consensus.cc:2781] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:32:14.037467 22229 raft_consensus.cc:482] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:32:14.037501 22229 raft_consensus.cc:3037] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:14.038232 22229 raft_consensus.cc:504] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:14.038410 22229 leader_election.cc:288] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 [CANDIDATE]: Term 1 election: Requested vote from peers 727655effcd8424f8eff83ad83d111b7 (127.19.238.65:35105), 3b65cb0ba707424c82542daf9dabe021 (127.19.238.67:46093)
I20260812 16:32:14.038584 22140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "149787aca59d48d09d60b1152673f30a" candidate_uuid: "0f97c49dd136430e8641ca1376b86e09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b65cb0ba707424c82542daf9dabe021"
I20260812 16:32:14.038597 21828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "149787aca59d48d09d60b1152673f30a" candidate_uuid: "0f97c49dd136430e8641ca1376b86e09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "727655effcd8424f8eff83ad83d111b7"
I20260812 16:32:14.038674 22140 raft_consensus.cc:3037] T 149787aca59d48d09d60b1152673f30a P 3b65cb0ba707424c82542daf9dabe021 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:14.038674 21828 raft_consensus.cc:3037] T 149787aca59d48d09d60b1152673f30a P 727655effcd8424f8eff83ad83d111b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:14.039497 22140 raft_consensus.cc:2454] T 149787aca59d48d09d60b1152673f30a P 3b65cb0ba707424c82542daf9dabe021 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f97c49dd136430e8641ca1376b86e09 in term 1.
I20260812 16:32:14.039497 21828 raft_consensus.cc:2454] T 149787aca59d48d09d60b1152673f30a P 727655effcd8424f8eff83ad83d111b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f97c49dd136430e8641ca1376b86e09 in term 1.
I20260812 16:32:14.039687 21907 leader_election.cc:302] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f97c49dd136430e8641ca1376b86e09, 3b65cb0ba707424c82542daf9dabe021; no voters:
I20260812 16:32:14.039793 22229 raft_consensus.cc:2781] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:14.039973 22229 raft_consensus.cc:686] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 [term 1 LEADER]: Becoming Leader. State: Replica: 0f97c49dd136430e8641ca1376b86e09, State: Running, Role: LEADER
I20260812 16:32:14.040032 22229 consensus_queue.cc:237] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } }
I20260812 16:32:14.040396 22268 tablet_replica.cc:439] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09: TxnStatusTablet state changed. Reason: New leader 0f97c49dd136430e8641ca1376b86e09. Latest consensus state: current_term: 1 leader_uuid: "0f97c49dd136430e8641ca1376b86e09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } } }
I20260812 16:32:14.040515 22268 tablet_replica.cc:442] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:32:14.040779 22270 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 16:32:14.040901 22270 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 16:32:14.040822 21652 catalog_manager.cc:5495] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f97c49dd136430e8641ca1376b86e09 (127.19.238.66). New cstate: current_term: 1 leader_uuid: "0f97c49dd136430e8641ca1376b86e09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } health_report { overall_health: UNKNOWN } } }
I20260812 16:32:14.050339 22219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:14.047621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53634 (local address 127.19.238.66:43977)
0812 16:32:14.047703 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:14.047705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:14.047715 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:14.047748 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:14.047749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:14.047765 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:14.047939 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:14.048102 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.048584 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.049353 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.049680 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.049760 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:14.049970 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:14.050085 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:14.050172 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:14.050240 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 16:32:14.050265 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 16:32:14.050379 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:14.046003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:43977 (local address 127.0.0.1:53634)
0812 16:32:14.046081 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:14.046098 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:14.046135 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:14.047959 (+ 1824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:14.047960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:14.047961 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:32:14.048016 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:14.048018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.048673 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.048675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:14.049260 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:14.049262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.049761 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.049762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:14.049830 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:14.049858 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:32:14.050156 (+ 298us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:32:14.050168 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 16:32:14.050235 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 16:32:14.050293 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 16:32:14.051753 21828 raft_consensus.cc:1261] T 149787aca59d48d09d60b1152673f30a P 727655effcd8424f8eff83ad83d111b7 [term 1 FOLLOWER]: Refusing update from remote peer 0f97c49dd136430e8641ca1376b86e09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:14.052080 22140 raft_consensus.cc:1261] T 149787aca59d48d09d60b1152673f30a P 3b65cb0ba707424c82542daf9dabe021 [term 1 FOLLOWER]: Refusing update from remote peer 0f97c49dd136430e8641ca1376b86e09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:14.052134 22268 consensus_queue.cc:1035] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:14.052348 22229 consensus_queue.cc:1035] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:14.052973 22232 tablet_replica.cc:439] T 149787aca59d48d09d60b1152673f30a P 727655effcd8424f8eff83ad83d111b7: TxnStatusTablet state changed. Reason: New leader 0f97c49dd136430e8641ca1376b86e09. Latest consensus state: current_term: 1 leader_uuid: "0f97c49dd136430e8641ca1376b86e09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } } }
I20260812 16:32:14.053064 22232 tablet_replica.cc:442] T 149787aca59d48d09d60b1152673f30a P 727655effcd8424f8eff83ad83d111b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:14.053001 22228 tablet_replica.cc:439] T 149787aca59d48d09d60b1152673f30a P 3b65cb0ba707424c82542daf9dabe021: TxnStatusTablet state changed. Reason: New leader 0f97c49dd136430e8641ca1376b86e09. Latest consensus state: current_term: 1 leader_uuid: "0f97c49dd136430e8641ca1376b86e09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } } }
I20260812 16:32:14.053098 22228 tablet_replica.cc:442] T 149787aca59d48d09d60b1152673f30a P 3b65cb0ba707424c82542daf9dabe021: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:14.053839 22268 tablet_replica.cc:439] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f97c49dd136430e8641ca1376b86e09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } } }
I20260812 16:32:14.054018 22268 tablet_replica.cc:442] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:32:14.054320 22268 tablet_replica.cc:439] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f97c49dd136430e8641ca1376b86e09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } } }
I20260812 16:32:14.054351 22232 tablet_replica.cc:439] T 149787aca59d48d09d60b1152673f30a P 727655effcd8424f8eff83ad83d111b7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f97c49dd136430e8641ca1376b86e09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } } }
I20260812 16:32:14.054430 22232 tablet_replica.cc:442] T 149787aca59d48d09d60b1152673f30a P 727655effcd8424f8eff83ad83d111b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:14.054525 22268 tablet_replica.cc:442] T 149787aca59d48d09d60b1152673f30a P 0f97c49dd136430e8641ca1376b86e09: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:32:14.055011 22228 tablet_replica.cc:439] T 149787aca59d48d09d60b1152673f30a P 3b65cb0ba707424c82542daf9dabe021: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f97c49dd136430e8641ca1376b86e09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "727655effcd8424f8eff83ad83d111b7" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35105 } } peers { permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 } } peers { permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 } } }
I20260812 16:32:14.055117 22228 tablet_replica.cc:442] T 149787aca59d48d09d60b1152673f30a P 3b65cb0ba707424c82542daf9dabe021: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:14.061932 22219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:14.058671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:34755 (local address 127.19.238.66:43977)
0812 16:32:14.058715 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:14.058718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:14.058920 (+ 202us) server_negotiation.cc:407] Connection header received
0812 16:32:14.058947 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:14.058948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:14.058956 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:14.059006 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:14.059178 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.059996 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.061162 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.061359 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.061435 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:14.061657 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:14.061687 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:14.061750 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:14.061839 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 16:32:14.061871 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:32:14.061962 22278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:14.058634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:43977 (local address 127.19.238.65:34755)
0812 16:32:14.058815 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:14.058827 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:14.058896 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:14.059026 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:14.059027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:14.059030 (+ 3us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:32:14.059088 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:14.059089 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.060099 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.060100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:14.061067 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:14.061070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.061453 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.061454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:14.061520 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:14.061541 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:32:14.061761 (+ 220us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:32:14.061764 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:32:14.061815 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 16:32:14.061868 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260812 16:32:14.064803 21983 raft_consensus.cc:1261] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09 [term 1 FOLLOWER]: Refusing update from remote peer 727655effcd8424f8eff83ad83d111b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:14.065035 22140 raft_consensus.cc:1261] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021 [term 1 FOLLOWER]: Refusing update from remote peer 727655effcd8424f8eff83ad83d111b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:14.065251 22232 consensus_queue.cc:1035] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f97c49dd136430e8641ca1376b86e09" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:14.065397 22241 consensus_queue.cc:1035] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b65cb0ba707424c82542daf9dabe021" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:14.079547 22216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:14.076492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:51901 (local address 127.19.238.65:35105)
0812 16:32:14.076557 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:14.076559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:14.076680 (+ 121us) server_negotiation.cc:407] Connection header received
0812 16:32:14.076746 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:14.076747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:14.076757 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:14.076807 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:14.077090 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.077843 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.078699 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.079026 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.079095 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:14.079258 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:14.079296 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:14.079358 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:14.079446 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 16:32:14.079467 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 16:32:14.080236 22283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:14.076388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:35105 (local address 127.19.238.66:51901)
0812 16:32:14.076601 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:14.076613 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:14.076684 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:14.076816 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:14.076817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:14.076818 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:32:14.076935 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:14.076938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.077927 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.077928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:14.078612 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:14.078614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.079103 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.079104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:14.079159 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:14.079175 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:32:14.079398 (+ 223us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:32:14.079400 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:32:14.079434 (+ 34us) client_negotiation.cc:240] Negotiation successful
0812 16:32:14.079470 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":144,"threads_started":1}
I20260812 16:32:14.088910 21652 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:14.089056 21652 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:14.089704 21652 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b92cc799aa14dc393b4043378886752: Sending DeleteTablet for 3 replicas of tablet 4a586ae798654f0aac15d57262d2b1ca
I20260812 16:32:14.089977 22118 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a586ae798654f0aac15d57262d2b1ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:14 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:41536
I20260812 16:32:14.090104 22249 tablet_replica.cc:330] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021: stopping tablet replica
I20260812 16:32:14.090193 22249 raft_consensus.cc:2229] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:14.090237 22249 raft_consensus.cc:2258] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:14.090252 21963 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a586ae798654f0aac15d57262d2b1ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:14 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:53628
I20260812 16:32:14.090343 22247 tablet_replica.cc:330] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09: stopping tablet replica
I20260812 16:32:14.090390 22247 raft_consensus.cc:2229] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:14.090423 22247 raft_consensus.cc:2258] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:14.090735 22247 ts_tablet_manager.cc:1905] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:14.090873 22249 ts_tablet_manager.cc:1905] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:14.090919 21805 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a586ae798654f0aac15d57262d2b1ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:14 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:54004
I20260812 16:32:14.092458 22248 tablet_replica.cc:330] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7: stopping tablet replica
I20260812 16:32:14.092545 22248 raft_consensus.cc:2229] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:32:14.092620 22248 raft_consensus.cc:2258] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:14.092846 22249 ts_tablet_manager.cc:1918] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:32:14.092908 22249 log.cc:1196] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/4a586ae798654f0aac15d57262d2b1ca
I20260812 16:32:14.092989 22248 ts_tablet_manager.cc:1905] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:14.093166 22249 ts_tablet_manager.cc:1939] T 4a586ae798654f0aac15d57262d2b1ca P 3b65cb0ba707424c82542daf9dabe021: Deleting consensus metadata
I20260812 16:32:14.093559 22247 ts_tablet_manager.cc:1918] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:32:14.093616 22247 log.cc:1196] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/4a586ae798654f0aac15d57262d2b1ca
I20260812 16:32:14.093827 22247 ts_tablet_manager.cc:1939] T 4a586ae798654f0aac15d57262d2b1ca P 0f97c49dd136430e8641ca1376b86e09: Deleting consensus metadata
I20260812 16:32:14.094151 21634 catalog_manager.cc:4841] TS 0f97c49dd136430e8641ca1376b86e09 (127.19.238.66:43977): tablet 4a586ae798654f0aac15d57262d2b1ca (table test-table [id=08a9eef9ce9547dea78b2fd3a7b6c473]) successfully deleted
I20260812 16:32:14.094671 22248 ts_tablet_manager.cc:1918] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:32:14.094724 22248 log.cc:1196] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786552327177731-20409-0/minicluster-data/ts-0/wal/wals/4a586ae798654f0aac15d57262d2b1ca
I20260812 16:32:14.094947 22248 ts_tablet_manager.cc:1939] T 4a586ae798654f0aac15d57262d2b1ca P 727655effcd8424f8eff83ad83d111b7: Deleting consensus metadata
I20260812 16:32:14.095089 21636 catalog_manager.cc:4841] TS 3b65cb0ba707424c82542daf9dabe021 (127.19.238.67:46093): tablet 4a586ae798654f0aac15d57262d2b1ca (table test-table [id=08a9eef9ce9547dea78b2fd3a7b6c473]) successfully deleted
I20260812 16:32:14.095400 21635 catalog_manager.cc:4841] TS 727655effcd8424f8eff83ad83d111b7 (127.19.238.65:35105): tablet 4a586ae798654f0aac15d57262d2b1ca (table test-table [id=08a9eef9ce9547dea78b2fd3a7b6c473]) successfully deleted
Aug 12 16:32:14 dist-test-slave-mvvj krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552333, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.238.65@KRBTEST.COM
I20260812 16:32:14.102487 22216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:14.092707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54028 (local address 127.19.238.65:35105)
0812 16:32:14.092753 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:14.092755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:14.092765 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:14.092789 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:14.092789 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:14.092802 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:14.092865 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:14.093670 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.094367 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.095533 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.095729 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.095821 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:14.099191 (+ 3370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:14.099201 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:14.099201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:14.099209 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:14.101060 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:14.101547 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:14.101551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:14.101551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:14.101621 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:14.101865 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:14.101866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:14.101866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:14.102029 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:14.102194 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:14.102279 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 16:32:14.102367 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 16:32:14.102753 21870 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54028
I20260812 16:32:14.110584 22262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:14.103889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41708 (local address 127.19.238.126:38907)
0812 16:32:14.103964 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:14.103966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:14.104079 (+ 113us) server_negotiation.cc:407] Connection header received
0812 16:32:14.104115 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:14.104115 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:14.104130 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:14.104203 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:14.104683 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.105519 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.106230 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:14.106565 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:14.106640 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:14.107917 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:14.107919 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:14.107920 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:14.107929 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:14.109496 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:14.109829 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:14.109830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:14.109830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:14.109894 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:14.110076 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:14.110077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:14.110077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:14.110216 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:14.110320 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:14.110390 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 16:32:14.110481 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 16:32:14.110826 21652 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41708
I20260812 16:32:14.111541 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 21734
I20260812 16:32:14.126758 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 21887
I20260812 16:32:14.134444 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 22045
I20260812 16:32:14.140375 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 21616
2026-08-12T16:32:14Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2747 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:32:14 dist-test-slave-mvvj krb5kdc[22292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:32:14 dist-test-slave-mvvj krb5kdc[22292](info): set up 2 sockets
Aug 12 16:32:14 dist-test-slave-mvvj krb5kdc[22292](info): commencing operation
krb5kdc: starting...
W20260812 16:32:15.264114 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:32:15 dist-test-slave-mvvj krb5kdc[22292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:32:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:32:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:32:15.396580 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:40441
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:40441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:15.485256 22308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:15.485517 22308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:15.485556 22308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:15.486716 22308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:15.486783 22308 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 16:32:15.486809 22308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:15.486840 22308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:15.486871 22308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:15.486899 22308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:32:15.488519 22308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:40441
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:40441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:15.488798 22308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:15.489037 22308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:15.494071 22318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:15.494171 22316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:15.494079 22314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:15.494318 22308 server_base.cc:1034] running on GCE node
I20260812 16:32:15.494673 22308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:15.494971 22308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:15.496146 22308 hybrid_clock.cc:648] HybridClock initialized: now 1786552335496131 us; error 44 us; skew 500 ppm
Aug 12 16:32:15 dist-test-slave-mvvj krb5kdc[22292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552335, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:15.498857 22308 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:32:15.499337 22308 webserver.cc:460] Webserver started at http://127.19.238.126:41819/ using document root <none> and password file <none>
I20260812 16:32:15.499575 22308 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:15.499647 22308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:15.499809 22308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:15.500936 22308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "1df569e0e14c4b24afd45ea55fac891f"
format_stamp: "Formatted at 2026-08-12 16:32:15 on dist-test-slave-mvvj"
server_key: "bb1b0ae162a36c577dcbed8501163139"
server_key_iv: "c4221a043d5956dda756623946d340d4"
server_key_version: "encryptionkey@0"
I20260812 16:32:15.501346 22308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "1df569e0e14c4b24afd45ea55fac891f"
format_stamp: "Formatted at 2026-08-12 16:32:15 on dist-test-slave-mvvj"
server_key: "bb1b0ae162a36c577dcbed8501163139"
server_key_iv: "c4221a043d5956dda756623946d340d4"
server_key_version: "encryptionkey@0"
I20260812 16:32:15.502912 22308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:32:15.503762 22326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:15.503998 22308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:15.504136 22308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "1df569e0e14c4b24afd45ea55fac891f"
format_stamp: "Formatted at 2026-08-12 16:32:15 on dist-test-slave-mvvj"
server_key: "bb1b0ae162a36c577dcbed8501163139"
server_key_iv: "c4221a043d5956dda756623946d340d4"
server_key_version: "encryptionkey@0"
I20260812 16:32:15.504218 22308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:15.516566 22308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:15.518537 22308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:15.518702 22308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:15.523505 22308 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:40441
I20260812 16:32:15.523599 22396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:40441 every 8 connection(s)
I20260812 16:32:15.523979 22308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:32:15.524765 22397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:15.527100 22397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f: Bootstrap starting.
I20260812 16:32:15.527783 22397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:15.528117 22397 log.cc:824] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:15.528785 22397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f: No bootstrap required, opened a new log
I20260812 16:32:15.530891 22397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1df569e0e14c4b24afd45ea55fac891f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40441 } }
I20260812 16:32:15.531032 22397 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:15.531055 22397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1df569e0e14c4b24afd45ea55fac891f, State: Initialized, Role: FOLLOWER
I20260812 16:32:15.531196 22397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1df569e0e14c4b24afd45ea55fac891f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40441 } }
I20260812 16:32:15.531284 22397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:32:15.531314 22397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:32:15.531349 22397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:15.531924 22397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1df569e0e14c4b24afd45ea55fac891f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40441 } }
I20260812 16:32:15.532013 22397 leader_election.cc:302] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1df569e0e14c4b24afd45ea55fac891f; no voters:
I20260812 16:32:15.532137 22397 leader_election.cc:288] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:32:15.532204 22400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:15.532395 22400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f [term 1 LEADER]: Becoming Leader. State: Replica: 1df569e0e14c4b24afd45ea55fac891f, State: Running, Role: LEADER
I20260812 16:32:15.532430 22397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:15.532485 22400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1df569e0e14c4b24afd45ea55fac891f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40441 } }
I20260812 16:32:15.532773 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 22308
I20260812 16:32:15.532927 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:32:15.533152 20409 external_mini_cluster.cc:1428] Setting key 913120cb4889467d57e1c7af2b3c1b13
I20260812 16:32:15.533758 22401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1df569e0e14c4b24afd45ea55fac891f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1df569e0e14c4b24afd45ea55fac891f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40441 } } }
I20260812 16:32:15.533726 22402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1df569e0e14c4b24afd45ea55fac891f. Latest consensus state: current_term: 1 leader_uuid: "1df569e0e14c4b24afd45ea55fac891f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1df569e0e14c4b24afd45ea55fac891f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40441 } } }
I20260812 16:32:15.533850 22401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:15.533854 22402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f [sys.catalog]: This master's current role is: LEADER
W20260812 16:32:15.534454 22417 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:32:15.534521 22417 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:32:15.534584 22419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:32:15.534740 22419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:32:15.536468 22419 catalog_manager.cc:1347] Generated new cluster ID: 331475bb82d24561b296c5c8eab969cf
I20260812 16:32:15.536533 22419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:32:15 dist-test-slave-mvvj krb5kdc[22292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:15.544384 22420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:15.534079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45234 (local address 127.19.238.126:40441)
0812 16:32:15.534542 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:15.534551 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:15.534585 (+ 34us) server_negotiation.cc:407] Connection header received
0812 16:32:15.534706 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:15.534713 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:15.534768 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:15.534902 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:15.535712 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:15.536980 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:15.537786 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:15.538217 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:15.538355 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:15.540813 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:15.540822 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:15.540823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:15.540835 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:15.542628 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:15.543041 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:15.543050 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:15.543050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:15.543155 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:15.543387 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:15.543388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:15.543388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:15.543650 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:15.543865 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:15.543987 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 16:32:15.544141 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":83,"threads_started":1}
I20260812 16:32:15.552030 22419 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:32:15.552639 22419 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:15.554972 22419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f: Generated new TSK 0
I20260812 16:32:15.555173 22419 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
I20260812 16:32:15.591619 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:40441
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:15.701915 22425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:15.702147 22425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:15.702176 22425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:15.703493 22425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:15.703554 22425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:15.703578 22425 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 16:32:15.703590 22425 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:32:15.703609 22425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:32:15.705367 22425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.238.126:40441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:15.705657 22425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:15.705960 22425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:15.712013 22434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:15.711975 22431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:15.712051 22425 server_base.cc:1034] running on GCE node
W20260812 16:32:15.711992 22432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:15.712472 22425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:15.712769 22425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:15.713945 22425 hybrid_clock.cc:648] HybridClock initialized: now 1786552335713945 us; error 43 us; skew 500 ppm
Aug 12 16:32:15 dist-test-slave-mvvj krb5kdc[22292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552335, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:15.716856 22425 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:32:15.717387 22425 webserver.cc:460] Webserver started at http://127.19.238.65:37755/ using document root <none> and password file <none>
I20260812 16:32:15.717613 22425 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:15.717676 22425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:15.717774 22425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:15.718933 22425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "62a0236c035643e4a8686166beec2b86"
format_stamp: "Formatted at 2026-08-12 16:32:15 on dist-test-slave-mvvj"
server_key: "f619945c0835431e44dc7c96dc980fa2"
server_key_iv: "fea1b0d55ef6e1a2a85e057369d9a5c6"
server_key_version: "encryptionkey@0"
I20260812 16:32:15.719262 22425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "62a0236c035643e4a8686166beec2b86"
format_stamp: "Formatted at 2026-08-12 16:32:15 on dist-test-slave-mvvj"
server_key: "f619945c0835431e44dc7c96dc980fa2"
server_key_iv: "fea1b0d55ef6e1a2a85e057369d9a5c6"
server_key_version: "encryptionkey@0"
I20260812 16:32:15.720717 22425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:32:15.721483 22441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:15.721740 22425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:15.721863 22425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "62a0236c035643e4a8686166beec2b86"
format_stamp: "Formatted at 2026-08-12 16:32:15 on dist-test-slave-mvvj"
server_key: "f619945c0835431e44dc7c96dc980fa2"
server_key_iv: "fea1b0d55ef6e1a2a85e057369d9a5c6"
server_key_version: "encryptionkey@0"
I20260812 16:32:15.721944 22425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:15.731047 22425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:15.733505 22425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:15.733625 22425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:15.734163 22425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:15.734210 22425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:15.734236 22425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:15.734247 22425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:15.741097 22425 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:35523
I20260812 16:32:15.741238 22569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:35523 every 8 connection(s)
I20260812 16:32:15.741524 22425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
Aug 12 16:32:15 dist-test-slave-mvvj krb5kdc[22292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552335, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:15.744997 22420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:15.734758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:51987 (local address 127.19.238.126:40441)
0812 16:32:15.734838 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:15.734840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:15.735064 (+ 224us) server_negotiation.cc:407] Connection header received
0812 16:32:15.735614 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:15.735616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:15.735638 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:15.735725 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:15.736682 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:15.737547 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:15.739156 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:15.739495 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:15.739601 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:15.742407 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:15.742409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:15.742409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:15.742418 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:15.743702 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:15.744140 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:15.744141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:15.744141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:15.744200 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:15.744428 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:15.744429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:15.744429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:15.744540 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:15.744589 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:15.744834 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 16:32:15.744926 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 16:32:15.745138 22451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:15.734711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:40441 (local address 127.19.238.65:51987)
0812 16:32:15.734921 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:15.734969 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:15.735483 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:15.735822 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:15.735827 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:15.736243 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:15.736552 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:15.736560 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:15.737659 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:15.737661 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:15.739048 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:15.739052 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:15.739605 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:15.739606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:15.739719 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:15.740409 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:15.740420 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:15.742293 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:15.743777 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:15.743784 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:15.743786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:15.744070 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:15.744262 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:15.744263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:15.744263 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:15.744364 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:15.744593 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:15.744598 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:15.744716 (+ 118us) client_negotiation.cc:769] Sending connection context
0812 16:32:15.744793 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 16:32:15.744917 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":124,"threads_started":1}
I20260812 16:32:15.745687 22570 heartbeater.cc:347] Connected to a master server at 127.19.238.126:40441
I20260812 16:32:15.745791 22570 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:15.745991 22570 heartbeater.cc:511] Master 127.19.238.126:40441 requested a full tablet report, sending...
I20260812 16:32:15.746560 22343 ts_manager.cc:194] Registered new tserver with Master: 62a0236c035643e4a8686166beec2b86 (127.19.238.65:35523)
I20260812 16:32:15.747181 22343 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:51987
I20260812 16:32:15.748670 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 22425
I20260812 16:32:15.748750 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:32:15.748973 20409 external_mini_cluster.cc:1428] Setting key dc33be76221f69346ef656bcf6b22588
I20260812 16:32:15.753912 22576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:15.747535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:40441 (local address 127.19.238.65:60329)
0812 16:32:15.747737 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:15.747748 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:15.747814 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:15.747989 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:15.747990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:15.748193 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:15.748263 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:15.748265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:15.749003 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:15.749004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:15.749578 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:15.749580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:15.749944 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:15.749945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:15.750025 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:15.750440 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:15.750445 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:15.750860 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:15.752820 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:15.752821 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:15.752822 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:15.753013 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:15.753248 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:15.753249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:15.753249 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:15.753320 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:15.753633 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:15.753634 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:15.753677 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 16:32:15.753736 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 16:32:15.753823 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":97,"threads_started":1}
I20260812 16:32:15.754012 22420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:15.747614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:60329 (local address 127.19.238.126:40441)
0812 16:32:15.747662 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:15.747665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:15.747817 (+ 152us) server_negotiation.cc:407] Connection header received
0812 16:32:15.747902 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:15.747903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:15.747913 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:15.747970 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:15.748341 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:15.748906 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:15.749649 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:15.749867 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:15.749919 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:15.750958 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:15.750960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:15.750960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:15.750966 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:15.752728 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:15.753089 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:15.753091 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:15.753091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:15.753156 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:15.753402 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:15.753403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:15.753403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:15.753550 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:15.753622 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:15.753781 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 16:32:15.753875 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for kudu/127.19.238.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
I20260812 16:32:15.777110 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:40441
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:15.863620 22581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:15.863898 22581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:15.863929 22581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:15.865221 22581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:15.865278 22581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:15.865301 22581 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 16:32:15.865314 22581 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:32:15.865331 22581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:32:15.866847 22581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.238.126:40441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:15.867064 22581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:15.867254 22581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:15.872578 22586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:15.872632 22587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:15.872605 22589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:15.872822 22581 server_base.cc:1034] running on GCE node
I20260812 16:32:15.873073 22581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:15.873330 22581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:15.874502 22581 hybrid_clock.cc:648] HybridClock initialized: now 1786552335874485 us; error 47 us; skew 500 ppm
Aug 12 16:32:15 dist-test-slave-mvvj krb5kdc[22292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552335, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:15.877166 22581 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:32:15.877630 22581 webserver.cc:460] Webserver started at http://127.19.238.66:36575/ using document root <none> and password file <none>
I20260812 16:32:15.877873 22581 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:15.877944 22581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:15.878100 22581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:15.879405 22581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "da0ad6333fd24309a6c43788ae9c702d"
format_stamp: "Formatted at 2026-08-12 16:32:15 on dist-test-slave-mvvj"
server_key: "f1e0404f8dcd77866f630f5ba6a04224"
server_key_iv: "1664a1b336e8fb2e26c1f80b650e9b9f"
server_key_version: "encryptionkey@0"
I20260812 16:32:15.879822 22581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "da0ad6333fd24309a6c43788ae9c702d"
format_stamp: "Formatted at 2026-08-12 16:32:15 on dist-test-slave-mvvj"
server_key: "f1e0404f8dcd77866f630f5ba6a04224"
server_key_iv: "1664a1b336e8fb2e26c1f80b650e9b9f"
server_key_version: "encryptionkey@0"
I20260812 16:32:15.881265 22581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:15.882211 22597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:15.882507 22581 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:15.882623 22581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "da0ad6333fd24309a6c43788ae9c702d"
format_stamp: "Formatted at 2026-08-12 16:32:15 on dist-test-slave-mvvj"
server_key: "f1e0404f8dcd77866f630f5ba6a04224"
server_key_iv: "1664a1b336e8fb2e26c1f80b650e9b9f"
server_key_version: "encryptionkey@0"
I20260812 16:32:15.882702 22581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:15.892030 22581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:15.894148 22581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:15.894304 22581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:15.894865 22581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:15.894932 22581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:15.894980 22581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:15.895005 22581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:15.901490 22581 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:41375
I20260812 16:32:15.901676 22728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:41375 every 8 connection(s)
I20260812 16:32:15.902001 22581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
Aug 12 16:32:15 dist-test-slave-mvvj krb5kdc[22292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552335, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:15.902829 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 22581
I20260812 16:32:15.902911 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:32:15.903069 20409 external_mini_cluster.cc:1428] Setting key dbca6a65a7e75dac454925718c8a680e
I20260812 16:32:15.906203 22420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:15.895423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:53613 (local address 127.19.238.126:40441)
0812 16:32:15.895501 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:15.895504 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:15.895801 (+ 297us) server_negotiation.cc:407] Connection header received
0812 16:32:15.896472 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:15.896474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:15.896488 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:15.896575 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:15.897416 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:15.898269 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:15.899825 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:15.900199 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:15.900276 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:15.903160 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:15.903161 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:15.903162 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:15.903171 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:15.904683 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:15.905121 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:15.905123 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:15.905123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:15.905201 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:15.905486 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:15.905488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:15.905488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:15.905626 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:15.905702 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:15.906012 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 16:32:15.906102 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 16:32:15.906417 22608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:15.895461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:40441 (local address 127.19.238.66:53613)
0812 16:32:15.895663 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:15.895704 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:15.896327 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:15.896652 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:15.896656 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:15.897026 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:15.897305 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:15.897311 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:15.898394 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:15.898395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:15.899731 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:15.899734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:15.900288 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:15.900289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:15.900449 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:15.901106 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:15.901116 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:15.903038 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:15.904778 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:15.904788 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:15.904790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:15.905037 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:15.905283 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:15.905283 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:15.905284 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:15.905413 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:15.905704 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:15.905712 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:15.905868 (+ 156us) client_negotiation.cc:769] Sending connection context
0812 16:32:15.905976 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 16:32:15.906153 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":133,"threads_started":1}
I20260812 16:32:15.907047 22730 heartbeater.cc:347] Connected to a master server at 127.19.238.126:40441
I20260812 16:32:15.907150 22730 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:15.907353 22730 heartbeater.cc:511] Master 127.19.238.126:40441 requested a full tablet report, sending...
I20260812 16:32:15.907733 22343 ts_manager.cc:194] Registered new tserver with Master: da0ad6333fd24309a6c43788ae9c702d (127.19.238.66:41375)
I20260812 16:32:15.908337 22343 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:53613
WARNING: no policy specified for kudu/127.19.238.67@KRBTEST.COM; defaulting to no policy
I20260812 16:32:15.915504 22738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:15.908648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:40441 (local address 127.19.238.66:51849)
0812 16:32:15.909399 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:15.909410 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:15.909465 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:15.909631 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:15.909632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:15.909793 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:15.909850 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:15.909852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:15.910843 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:15.910844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:15.911378 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:15.911380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:15.911925 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:15.911926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:15.912005 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:15.912505 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:15.912509 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:15.912783 (+ 274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:15.914347 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:15.914348 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:15.914349 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:15.914616 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:15.914822 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:15.914823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:15.914823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:15.914886 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:15.915191 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:15.915193 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:15.915241 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 16:32:15.915311 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 16:32:15.915424 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"thread_start_us":685,"threads_started":1}
I20260812 16:32:15.915668 22420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:15.908686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:51849 (local address 127.19.238.126:40441)
0812 16:32:15.908725 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:15.908727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:15.909508 (+ 781us) server_negotiation.cc:407] Connection header received
0812 16:32:15.909545 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:15.909547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:15.909560 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:15.909612 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:15.909927 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:15.910734 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:15.911468 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:15.911793 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:15.911927 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:15.912873 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:15.912874 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:15.912875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:15.912882 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:15.914245 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:15.914696 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:15.914697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:15.914697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:15.914754 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:15.914946 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:15.914946 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:15.914947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:15.915099 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:15.915156 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:15.915342 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 16:32:15.915411 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
I20260812 16:32:15.933172 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:40441
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:16.017984 22740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:16.018273 22740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:16.018345 22740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:16.019549 22740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:16.019593 22740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:16.019646 22740 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 16:32:16.019668 22740 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:32:16.019691 22740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:32:16.021311 22740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.238.126:40441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:16.021541 22740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:16.021729 22740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:16.027611 22747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:16.027674 22746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:16.027688 22749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:16.028211 22740 server_base.cc:1034] running on GCE node
I20260812 16:32:16.028462 22740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:16.028765 22740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:16.029915 22740 hybrid_clock.cc:648] HybridClock initialized: now 1786552336029902 us; error 29 us; skew 500 ppm
Aug 12 16:32:16 dist-test-slave-mvvj krb5kdc[22292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552336, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:16.032688 22740 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:32:16.033281 22740 webserver.cc:460] Webserver started at http://127.19.238.67:32947/ using document root <none> and password file <none>
I20260812 16:32:16.033579 22740 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:16.033658 22740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:16.033802 22740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:16.034965 22740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "d7a59ddbded945de918ffa6937ea91e3"
format_stamp: "Formatted at 2026-08-12 16:32:16 on dist-test-slave-mvvj"
server_key: "418e65285bb7e55171d25f48c0570f18"
server_key_iv: "1a7e8d78d5ff8f26f1846c3554791eb8"
server_key_version: "encryptionkey@0"
I20260812 16:32:16.035357 22740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "d7a59ddbded945de918ffa6937ea91e3"
format_stamp: "Formatted at 2026-08-12 16:32:16 on dist-test-slave-mvvj"
server_key: "418e65285bb7e55171d25f48c0570f18"
server_key_iv: "1a7e8d78d5ff8f26f1846c3554791eb8"
server_key_version: "encryptionkey@0"
I20260812 16:32:16.036863 22740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:32:16.037648 22756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:16.037818 22740 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:16.037919 22740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "d7a59ddbded945de918ffa6937ea91e3"
format_stamp: "Formatted at 2026-08-12 16:32:16 on dist-test-slave-mvvj"
server_key: "418e65285bb7e55171d25f48c0570f18"
server_key_iv: "1a7e8d78d5ff8f26f1846c3554791eb8"
server_key_version: "encryptionkey@0"
I20260812 16:32:16.038005 22740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:16.053159 22740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:16.055662 22740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:16.055907 22740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:16.056542 22740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:16.056604 22740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:16.056648 22740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:16.056674 22740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 16:32:16 dist-test-slave-mvvj krb5kdc[22292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552336, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:16.064268 22740 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:44449
I20260812 16:32:16.064396 22884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:44449 every 8 connection(s)
I20260812 16:32:16.064707 22740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:32:16.067149 22420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.057021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:57909 (local address 127.19.238.126:40441)
0812 16:32:16.057065 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:16.057068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:16.057267 (+ 199us) server_negotiation.cc:407] Connection header received
0812 16:32:16.057815 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:16.057816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:16.057832 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:16.057912 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:16.058663 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.059538 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.060761 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.061113 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.061201 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.064068 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:16.064070 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:16.064070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:16.064079 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:16.065736 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:16.066147 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:16.066149 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:16.066149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:16.066212 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:16.066490 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:16.066491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:16.066491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:16.066614 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:16.066692 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:16.066972 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 16:32:16.067056 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:32:16.067344 22765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.057001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:40441 (local address 127.19.238.67:57909)
0812 16:32:16.057153 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:16.057191 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:16.057702 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:16.057995 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:16.057999 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:16.058247 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:16.058573 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.058579 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.059660 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.059662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.060685 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.060688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.061225 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.061226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.061368 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.061955 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:16.061965 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:16.063955 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:16.065828 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:16.065835 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:16.065837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:16.066063 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:16.066288 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:16.066289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:16.066289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:16.066404 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:16.066684 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:16.066691 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:16.066831 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 16:32:16.066944 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 16:32:16.067109 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260812 16:32:16.067925 22885 heartbeater.cc:347] Connected to a master server at 127.19.238.126:40441
I20260812 16:32:16.068012 22885 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:16.068205 22885 heartbeater.cc:511] Master 127.19.238.126:40441 requested a full tablet report, sending...
I20260812 16:32:16.068571 22344 ts_manager.cc:194] Registered new tserver with Master: d7a59ddbded945de918ffa6937ea91e3 (127.19.238.67:44449)
I20260812 16:32:16.069113 22344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:57909
I20260812 16:32:16.069128 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 22740
I20260812 16:32:16.069231 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:32:16.069446 20409 external_mini_cluster.cc:1428] Setting key 6ba44f02719dcf7b5bf87562ea7d2532
I20260812 16:32:16.070343 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:32:16.075589 22420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.069050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:35983 (local address 127.19.238.126:40441)
0812 16:32:16.069098 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:16.069100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:16.069237 (+ 137us) server_negotiation.cc:407] Connection header received
0812 16:32:16.069367 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:16.069368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:16.069379 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:16.069438 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:16.069900 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.070535 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.071544 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.071719 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.071768 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.072855 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:16.072856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:16.072857 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:16.072866 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:16.074390 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:16.074848 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:16.074849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:16.074850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:16.074902 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:16.075117 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:16.075118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:16.075118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:16.075217 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:16.075269 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:16.075453 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 16:32:16.075525 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 16:32:16.075726 22892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.068978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:40441 (local address 127.19.238.67:35983)
0812 16:32:16.069166 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:16.069178 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:16.069264 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:16.069444 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:16.069445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:16.069674 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:16.069753 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.069755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.070613 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.070614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.071433 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.071436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.071791 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.071792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.071902 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.072316 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:16.072322 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:16.072749 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:16.074514 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:16.074515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:16.074516 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:16.074771 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:16.074970 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:16.074971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:16.074971 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:16.075053 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:16.075283 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:16.075284 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:16.075338 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 16:32:16.075434 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 16:32:16.075567 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":64,"threads_started":1}
Aug 12 16:32:16 dist-test-slave-mvvj krb5kdc[22292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 16:32:16 dist-test-slave-mvvj krb5kdc[22292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:16.096899 22420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.089736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45240 (local address 127.19.238.126:40441)
0812 16:32:16.089811 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:16.089813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:16.089822 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:32:16.089856 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:16.089857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:16.089867 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:16.089938 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:16.090545 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.091049 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.091810 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.092059 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.092137 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.094439 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:16.094440 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:16.094441 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:16.094455 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:16.095689 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:16.096085 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:16.096086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:16.096087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:16.096147 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:16.096362 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:16.096363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:16.096363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:16.096484 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:16.096594 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:16.096704 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 16:32:16.096788 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 16:32:16.098491 22344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:32:16.100088 22344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:32:16.106976 22903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.102552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:41375 (local address 127.0.0.1:49518)
0812 16:32:16.102908 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:16.102920 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:16.103001 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:16.103375 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:16.103376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:16.103378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:16.103411 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.103413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.104653 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.104655 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.105659 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.105661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.106642 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.106643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.106726 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.106741 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 16:32:16.106815 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 16:32:16.106876 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":93,"threads_started":1}
I20260812 16:32:16.106980 22907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.102754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:35523 (local address 127.0.0.1:33156)
0812 16:32:16.102973 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:16.102984 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:16.103031 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:16.103449 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:16.103450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:16.103456 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:16.103504 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.103506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.104475 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.104476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.105841 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.105844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.106743 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.106744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.106814 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.106817 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:32:16.106863 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 16:32:16.106894 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":68,"thread_start_us":69,"threads_started":1}
I20260812 16:32:16.107225 22909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.102967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33156 (local address 127.19.238.65:35523)
0812 16:32:16.103198 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:16.103201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:16.103229 (+ 28us) server_negotiation.cc:407] Connection header received
0812 16:32:16.103270 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:16.103272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:16.103288 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:16.103363 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:16.103586 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.104383 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.105950 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.106561 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.106652 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.106711 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:16.107055 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 16:32:16.107104 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":116,"threads_started":1}
I20260812 16:32:16.107049 22905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.102649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49518 (local address 127.19.238.66:41375)
0812 16:32:16.103161 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:16.103165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:16.103175 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:16.103212 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:16.103214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:16.103229 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:16.103303 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:16.103488 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.104581 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.105749 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.106554 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.106632 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.106699 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:16.106844 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 16:32:16.106887 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":88,"threads_started":1}
I20260812 16:32:16.107975 22904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.102546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:44449 (local address 127.0.0.1:46210)
0812 16:32:16.102705 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:16.102749 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:16.102813 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:16.103151 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:16.103154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:16.103157 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:16.103229 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.103231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.104602 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.104603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.105841 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.105844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.106811 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.106812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.106893 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.106898 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:32:16.106952 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 16:32:16.107002 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":86,"thread_start_us":50,"threads_started":1}
I20260812 16:32:16.108000 22906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.102741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46210 (local address 127.19.238.67:44449)
0812 16:32:16.102889 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:16.102894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:16.102906 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:32:16.102956 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:16.102959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:16.102977 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:16.103059 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:16.103325 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.104520 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.105933 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.106645 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.106707 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.106756 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:16.107065 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 16:32:16.107904 (+ 839us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260812 16:32:16.108094 22498 tablet_service.cc:1467] Processing CreateTablet for tablet 0739c167345645aaae7ffb7ad1fdb3a1 (DEFAULT_TABLE table=test-table [id=fb00e4e38e014350aaa96a49e429f986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:16.108095 22653 tablet_service.cc:1467] Processing CreateTablet for tablet 0739c167345645aaae7ffb7ad1fdb3a1 (DEFAULT_TABLE table=test-table [id=fb00e4e38e014350aaa96a49e429f986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:16.108498 22498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0739c167345645aaae7ffb7ad1fdb3a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:16.108498 22653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0739c167345645aaae7ffb7ad1fdb3a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:16.112392 22911 tablet_bootstrap.cc:492] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d: Bootstrap starting.
I20260812 16:32:16.112865 22812 tablet_service.cc:1467] Processing CreateTablet for tablet 0739c167345645aaae7ffb7ad1fdb3a1 (DEFAULT_TABLE table=test-table [id=fb00e4e38e014350aaa96a49e429f986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:16.113041 22911 tablet_bootstrap.cc:654] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:16.113211 22812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0739c167345645aaae7ffb7ad1fdb3a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:16.113391 22911 log.cc:824] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:16.113761 22910 tablet_bootstrap.cc:492] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86: Bootstrap starting.
I20260812 16:32:16.114221 22911 tablet_bootstrap.cc:492] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d: No bootstrap required, opened a new log
I20260812 16:32:16.114305 22911 ts_tablet_manager.cc:1397] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:32:16.114389 22910 tablet_bootstrap.cc:654] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:16.114677 22910 log.cc:824] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:16.115365 22910 tablet_bootstrap.cc:492] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86: No bootstrap required, opened a new log
I20260812 16:32:16.115445 22910 ts_tablet_manager.cc:1397] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:32:16.116462 22911 raft_consensus.cc:348] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } } peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } }
I20260812 16:32:16.116642 22911 raft_consensus.cc:374] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:16.116690 22911 raft_consensus.cc:729] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da0ad6333fd24309a6c43788ae9c702d, State: Initialized, Role: FOLLOWER
I20260812 16:32:16.116833 22911 consensus_queue.cc:260] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } } peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } }
I20260812 16:32:16.116863 22914 tablet_bootstrap.cc:492] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3: Bootstrap starting.
I20260812 16:32:16.117131 22911 ts_tablet_manager.cc:1428] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:32:16.117116 22910 raft_consensus.cc:348] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } } peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } }
I20260812 16:32:16.117290 22730 heartbeater.cc:503] Master 127.19.238.126:40441 was elected leader, sending a full tablet report...
I20260812 16:32:16.117298 22910 raft_consensus.cc:374] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:16.117339 22910 raft_consensus.cc:729] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a0236c035643e4a8686166beec2b86, State: Initialized, Role: FOLLOWER
I20260812 16:32:16.117458 22910 consensus_queue.cc:260] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } } peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } }
I20260812 16:32:16.117697 22910 ts_tablet_manager.cc:1428] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 16:32:16.117738 22570 heartbeater.cc:503] Master 127.19.238.126:40441 was elected leader, sending a full tablet report...
I20260812 16:32:16.117777 22914 tablet_bootstrap.cc:654] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:16.118126 22914 log.cc:824] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:16.118877 22914 tablet_bootstrap.cc:492] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3: No bootstrap required, opened a new log
I20260812 16:32:16.118961 22914 ts_tablet_manager.cc:1397] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:32:16.120492 22914 raft_consensus.cc:348] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } } peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } }
I20260812 16:32:16.120617 22914 raft_consensus.cc:374] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:16.120638 22914 raft_consensus.cc:729] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a59ddbded945de918ffa6937ea91e3, State: Initialized, Role: FOLLOWER
I20260812 16:32:16.120769 22914 consensus_queue.cc:260] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } } peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } }
I20260812 16:32:16.121031 22914 ts_tablet_manager.cc:1428] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:32:16.121097 22885 heartbeater.cc:503] Master 127.19.238.126:40441 was elected leader, sending a full tablet report...
W20260812 16:32:16.152702 22731 tablet.cc:2369] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:32:16.224093 22916 raft_consensus.cc:482] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:32:16.224257 22916 raft_consensus.cc:504] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } } peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } }
I20260812 16:32:16.224737 22916 leader_election.cc:288] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da0ad6333fd24309a6c43788ae9c702d (127.19.238.66:41375), d7a59ddbded945de918ffa6937ea91e3 (127.19.238.67:44449)
I20260812 16:32:16.228694 22919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.224955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:44449 (local address 127.19.238.65:46553)
0812 16:32:16.225201 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:16.225211 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:16.225297 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:16.225480 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:16.225481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:16.225482 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:16.225535 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.225536 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.226396 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.226396 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.227561 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.227565 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.228368 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.228369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.228453 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.228458 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:32:16.228538 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 16:32:16.228586 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":106,"thread_start_us":110,"threads_started":1}
I20260812 16:32:16.228685 22451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.224955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:41375 (local address 127.19.238.65:54555)
0812 16:32:16.225012 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:16.225026 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:16.225089 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:16.225287 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:16.225288 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:16.225289 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:16.225348 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.225350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.226299 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.226300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.227561 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.227565 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.228445 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.228446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.228531 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.228534 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:32:16.228587 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 16:32:16.228615 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":54}
I20260812 16:32:16.228792 22906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.225101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:46553 (local address 127.19.238.67:44449)
0812 16:32:16.225171 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:16.225173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:16.225273 (+ 100us) server_negotiation.cc:407] Connection header received
0812 16:32:16.225396 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:16.225397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:16.225408 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:16.225463 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:16.225603 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.226312 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.227673 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.228286 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.228362 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.228372 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:16.228551 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 16:32:16.228613 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 16:32:16.228792 22905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.225044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:54555 (local address 127.19.238.66:41375)
0812 16:32:16.225093 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:16.225096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:16.225134 (+ 38us) server_negotiation.cc:407] Connection header received
0812 16:32:16.225169 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:16.225170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:16.225179 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:16.225238 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:16.225431 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.226216 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.227674 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.228277 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.228440 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.228448 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:16.228601 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 16:32:16.228724 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:32:16.229105 22676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0739c167345645aaae7ffb7ad1fdb3a1" candidate_uuid: "62a0236c035643e4a8686166beec2b86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da0ad6333fd24309a6c43788ae9c702d" is_pre_election: true
I20260812 16:32:16.229230 22676 raft_consensus.cc:2454] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a0236c035643e4a8686166beec2b86 in term 0.
I20260812 16:32:16.229166 22833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0739c167345645aaae7ffb7ad1fdb3a1" candidate_uuid: "62a0236c035643e4a8686166beec2b86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a59ddbded945de918ffa6937ea91e3" is_pre_election: true
I20260812 16:32:16.229324 22833 raft_consensus.cc:2454] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a0236c035643e4a8686166beec2b86 in term 0.
I20260812 16:32:16.229511 22443 leader_election.cc:302] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a0236c035643e4a8686166beec2b86, da0ad6333fd24309a6c43788ae9c702d; no voters:
I20260812 16:32:16.229642 22916 raft_consensus.cc:2781] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:32:16.229686 22916 raft_consensus.cc:482] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:32:16.229707 22916 raft_consensus.cc:3037] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:16.230433 22916 raft_consensus.cc:504] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } } peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } }
I20260812 16:32:16.230584 22916 leader_election.cc:288] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [CANDIDATE]: Term 1 election: Requested vote from peers da0ad6333fd24309a6c43788ae9c702d (127.19.238.66:41375), d7a59ddbded945de918ffa6937ea91e3 (127.19.238.67:44449)
I20260812 16:32:16.230739 22676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0739c167345645aaae7ffb7ad1fdb3a1" candidate_uuid: "62a0236c035643e4a8686166beec2b86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da0ad6333fd24309a6c43788ae9c702d"
I20260812 16:32:16.230818 22833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0739c167345645aaae7ffb7ad1fdb3a1" candidate_uuid: "62a0236c035643e4a8686166beec2b86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a59ddbded945de918ffa6937ea91e3"
I20260812 16:32:16.230830 22676 raft_consensus.cc:3037] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:16.230892 22833 raft_consensus.cc:3037] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:16.231575 22833 raft_consensus.cc:2454] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a0236c035643e4a8686166beec2b86 in term 1.
I20260812 16:32:16.231601 22676 raft_consensus.cc:2454] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a0236c035643e4a8686166beec2b86 in term 1.
I20260812 16:32:16.231763 22443 leader_election.cc:302] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a0236c035643e4a8686166beec2b86, da0ad6333fd24309a6c43788ae9c702d; no voters:
I20260812 16:32:16.231891 22916 raft_consensus.cc:2781] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:16.232112 22916 raft_consensus.cc:686] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [term 1 LEADER]: Becoming Leader. State: Replica: 62a0236c035643e4a8686166beec2b86, State: Running, Role: LEADER
I20260812 16:32:16.232223 22916 consensus_queue.cc:237] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } } peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } }
I20260812 16:32:16.233119 22343 catalog_manager.cc:5495] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62a0236c035643e4a8686166beec2b86 (127.19.238.65). New cstate: current_term: 1 leader_uuid: "62a0236c035643e4a8686166beec2b86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } health_report { overall_health: HEALTHY } } }
W20260812 16:32:16.242326 22571 tablet.cc:2369] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 16:32:16 dist-test-slave-mvvj krb5kdc[22292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552335, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:16.280010 22420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.272721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38548 (local address 127.19.238.126:40441)
0812 16:32:16.272810 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:16.272816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:16.272873 (+ 57us) server_negotiation.cc:407] Connection header received
0812 16:32:16.272935 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:16.272936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:16.272948 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:16.273004 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:16.273561 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.274055 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.274812 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.274996 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.275059 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.277292 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:16.277294 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:16.277294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:16.277300 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:16.278802 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:16.279209 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:16.279211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:16.279211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:16.279270 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:16.279481 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:16.279482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:16.279482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:16.279607 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:16.279673 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:16.279799 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 16:32:16.279943 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 16:32:16.280263 22927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.272621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:40441 (local address 127.0.0.1:38548)
0812 16:32:16.272794 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:16.272807 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:16.272878 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:16.273045 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:16.273047 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:16.273432 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:16.273484 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.273487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.274145 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.274146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.274728 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.274731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.275071 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.275071 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.275113 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.275617 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:16.275622 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:16.277195 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:16.278907 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:16.278914 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:16.278916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:16.279121 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:16.279341 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:16.279342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:16.279342 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:16.279398 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:16.279677 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:16.279683 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:16.279726 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 16:32:16.279784 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 16:32:16.280023 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":142,"thread_start_us":83,"threads_started":1}
I20260812 16:32:16.282291 22343 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:38548:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 16:32:16.282505 22343 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:32:16.284171 22653 tablet_service.cc:1467] Processing CreateTablet for tablet 885fc40ec21b49b592b14df7fd226cbf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b95f16ced83640dcacfb1b87aff5057b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:32:16.284179 22498 tablet_service.cc:1467] Processing CreateTablet for tablet 885fc40ec21b49b592b14df7fd226cbf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b95f16ced83640dcacfb1b87aff5057b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:32:16.284358 22498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 885fc40ec21b49b592b14df7fd226cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:16.284359 22653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 885fc40ec21b49b592b14df7fd226cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:16.285797 22910 tablet_bootstrap.cc:492] T 885fc40ec21b49b592b14df7fd226cbf P 62a0236c035643e4a8686166beec2b86: Bootstrap starting.
I20260812 16:32:16.285856 22911 tablet_bootstrap.cc:492] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d: Bootstrap starting.
I20260812 16:32:16.285991 22812 tablet_service.cc:1467] Processing CreateTablet for tablet 885fc40ec21b49b592b14df7fd226cbf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b95f16ced83640dcacfb1b87aff5057b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:32:16.286151 22812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 885fc40ec21b49b592b14df7fd226cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:16.286294 22910 tablet_bootstrap.cc:654] T 885fc40ec21b49b592b14df7fd226cbf P 62a0236c035643e4a8686166beec2b86: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:16.286535 22911 tablet_bootstrap.cc:654] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:16.287178 22911 tablet_bootstrap.cc:492] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d: No bootstrap required, opened a new log
I20260812 16:32:16.287220 22911 ts_tablet_manager.cc:1397] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:16.287397 22911 raft_consensus.cc:348] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:16.287494 22911 raft_consensus.cc:374] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:16.287533 22911 raft_consensus.cc:729] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da0ad6333fd24309a6c43788ae9c702d, State: Initialized, Role: FOLLOWER
I20260812 16:32:16.287569 22911 consensus_queue.cc:260] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:16.287675 22911 ts_tablet_manager.cc:1428] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:16.287735 22915 tablet_replica.cc:439] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } } }
I20260812 16:32:16.287820 22915 tablet_replica.cc:442] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:16.288056 22914 tablet_bootstrap.cc:492] T 885fc40ec21b49b592b14df7fd226cbf P d7a59ddbded945de918ffa6937ea91e3: Bootstrap starting.
I20260812 16:32:16.288416 22910 tablet_bootstrap.cc:492] T 885fc40ec21b49b592b14df7fd226cbf P 62a0236c035643e4a8686166beec2b86: No bootstrap required, opened a new log
I20260812 16:32:16.288488 22910 ts_tablet_manager.cc:1397] T 885fc40ec21b49b592b14df7fd226cbf P 62a0236c035643e4a8686166beec2b86: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 16:32:16.288647 22914 tablet_bootstrap.cc:654] T 885fc40ec21b49b592b14df7fd226cbf P d7a59ddbded945de918ffa6937ea91e3: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:16.288653 22910 raft_consensus.cc:348] T 885fc40ec21b49b592b14df7fd226cbf P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:16.288727 22910 raft_consensus.cc:374] T 885fc40ec21b49b592b14df7fd226cbf P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:16.288764 22910 raft_consensus.cc:729] T 885fc40ec21b49b592b14df7fd226cbf P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a0236c035643e4a8686166beec2b86, State: Initialized, Role: FOLLOWER
I20260812 16:32:16.288821 22910 consensus_queue.cc:260] T 885fc40ec21b49b592b14df7fd226cbf P 62a0236c035643e4a8686166beec2b86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:16.288932 22910 ts_tablet_manager.cc:1428] T 885fc40ec21b49b592b14df7fd226cbf P 62a0236c035643e4a8686166beec2b86: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:16.289101 22916 tablet_replica.cc:439] T 885fc40ec21b49b592b14df7fd226cbf P 62a0236c035643e4a8686166beec2b86: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } } }
I20260812 16:32:16.289202 22916 tablet_replica.cc:442] T 885fc40ec21b49b592b14df7fd226cbf P 62a0236c035643e4a8686166beec2b86: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:16.289362 22914 tablet_bootstrap.cc:492] T 885fc40ec21b49b592b14df7fd226cbf P d7a59ddbded945de918ffa6937ea91e3: No bootstrap required, opened a new log
I20260812 16:32:16.289419 22914 ts_tablet_manager.cc:1397] T 885fc40ec21b49b592b14df7fd226cbf P d7a59ddbded945de918ffa6937ea91e3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:16.289618 22914 raft_consensus.cc:348] T 885fc40ec21b49b592b14df7fd226cbf P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:16.289718 22914 raft_consensus.cc:374] T 885fc40ec21b49b592b14df7fd226cbf P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:16.289744 22914 raft_consensus.cc:729] T 885fc40ec21b49b592b14df7fd226cbf P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a59ddbded945de918ffa6937ea91e3, State: Initialized, Role: FOLLOWER
I20260812 16:32:16.289795 22914 consensus_queue.cc:260] T 885fc40ec21b49b592b14df7fd226cbf P d7a59ddbded945de918ffa6937ea91e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:16.289897 22914 ts_tablet_manager.cc:1428] T 885fc40ec21b49b592b14df7fd226cbf P d7a59ddbded945de918ffa6937ea91e3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:16.290047 22918 tablet_replica.cc:439] T 885fc40ec21b49b592b14df7fd226cbf P d7a59ddbded945de918ffa6937ea91e3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } } }
I20260812 16:32:16.290139 22918 tablet_replica.cc:442] T 885fc40ec21b49b592b14df7fd226cbf P d7a59ddbded945de918ffa6937ea91e3: This TxnStatusTablet replica's current role is: FOLLOWER
W20260812 16:32:16.315568 22887 tablet.cc:2369] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:32:16.386286 22915 raft_consensus.cc:482] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:32:16.386410 22915 raft_consensus.cc:504] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:16.386920 22915 leader_election.cc:288] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62a0236c035643e4a8686166beec2b86 (127.19.238.65:35523), d7a59ddbded945de918ffa6937ea91e3 (127.19.238.67:44449)
I20260812 16:32:16.390115 22931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.387142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:44449 (local address 127.19.238.66:55517)
0812 16:32:16.387297 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:16.387307 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:16.387369 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:16.387583 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:16.387584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:16.387585 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:16.387687 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.387688 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.388488 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.388489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.389261 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.389263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.389847 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.389848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.389923 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.389929 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 16:32:16.389984 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 16:32:16.390049 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":83,"threads_started":1}
I20260812 16:32:16.390128 22906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.387251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:55517 (local address 127.19.238.67:44449)
0812 16:32:16.387310 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:16.387312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:16.387443 (+ 131us) server_negotiation.cc:407] Connection header received
0812 16:32:16.387483 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:16.387484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:16.387494 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:16.387572 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:16.387790 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.388408 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.389325 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.389744 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.389798 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.389806 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:16.390021 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 16:32:16.390060 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 16:32:16.390285 22608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.387104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:35523 (local address 127.19.238.66:57655)
0812 16:32:16.387192 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:16.387205 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:16.387256 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:16.387481 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:16.387482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:16.387485 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:16.387567 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.387570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.388334 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.388335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.389386 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.389389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.390011 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.390012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.390104 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.390108 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:16.390169 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 16:32:16.390213 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":3}
I20260812 16:32:16.390322 22909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.387209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:57655 (local address 127.19.238.65:35523)
0812 16:32:16.387272 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:16.387274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:16.387286 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:32:16.387326 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:16.387327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:16.387337 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:16.387401 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:16.387653 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.388260 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.389475 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.389900 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.389979 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.389988 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:16.390191 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 16:32:16.390241 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 16:32:16.390350 22833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "885fc40ec21b49b592b14df7fd226cbf" candidate_uuid: "da0ad6333fd24309a6c43788ae9c702d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a59ddbded945de918ffa6937ea91e3" is_pre_election: true
I20260812 16:32:16.390448 22833 raft_consensus.cc:2454] T 885fc40ec21b49b592b14df7fd226cbf P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da0ad6333fd24309a6c43788ae9c702d in term 0.
I20260812 16:32:16.390604 22522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "885fc40ec21b49b592b14df7fd226cbf" candidate_uuid: "da0ad6333fd24309a6c43788ae9c702d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a0236c035643e4a8686166beec2b86" is_pre_election: true
I20260812 16:32:16.390767 22522 raft_consensus.cc:2454] T 885fc40ec21b49b592b14df7fd226cbf P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da0ad6333fd24309a6c43788ae9c702d in term 0.
I20260812 16:32:16.390980 22598 leader_election.cc:302] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7a59ddbded945de918ffa6937ea91e3, da0ad6333fd24309a6c43788ae9c702d; no voters:
I20260812 16:32:16.391129 22915 raft_consensus.cc:2781] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:32:16.391180 22915 raft_consensus.cc:482] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:32:16.391212 22915 raft_consensus.cc:3037] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:16.391964 22915 raft_consensus.cc:504] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:16.392131 22915 leader_election.cc:288] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d [CANDIDATE]: Term 1 election: Requested vote from peers 62a0236c035643e4a8686166beec2b86 (127.19.238.65:35523), d7a59ddbded945de918ffa6937ea91e3 (127.19.238.67:44449)
I20260812 16:32:16.392314 22522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "885fc40ec21b49b592b14df7fd226cbf" candidate_uuid: "da0ad6333fd24309a6c43788ae9c702d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a0236c035643e4a8686166beec2b86"
I20260812 16:32:16.392335 22833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "885fc40ec21b49b592b14df7fd226cbf" candidate_uuid: "da0ad6333fd24309a6c43788ae9c702d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a59ddbded945de918ffa6937ea91e3"
I20260812 16:32:16.392405 22522 raft_consensus.cc:3037] T 885fc40ec21b49b592b14df7fd226cbf P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:16.392405 22833 raft_consensus.cc:3037] T 885fc40ec21b49b592b14df7fd226cbf P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:16.393213 22833 raft_consensus.cc:2454] T 885fc40ec21b49b592b14df7fd226cbf P d7a59ddbded945de918ffa6937ea91e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da0ad6333fd24309a6c43788ae9c702d in term 1.
I20260812 16:32:16.393213 22522 raft_consensus.cc:2454] T 885fc40ec21b49b592b14df7fd226cbf P 62a0236c035643e4a8686166beec2b86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da0ad6333fd24309a6c43788ae9c702d in term 1.
I20260812 16:32:16.393395 22598 leader_election.cc:302] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7a59ddbded945de918ffa6937ea91e3, da0ad6333fd24309a6c43788ae9c702d; no voters:
I20260812 16:32:16.393550 22915 raft_consensus.cc:2781] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:16.393725 22915 raft_consensus.cc:686] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d [term 1 LEADER]: Becoming Leader. State: Replica: da0ad6333fd24309a6c43788ae9c702d, State: Running, Role: LEADER
I20260812 16:32:16.393847 22915 consensus_queue.cc:237] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:16.394308 22932 tablet_replica.cc:439] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d: TxnStatusTablet state changed. Reason: New leader da0ad6333fd24309a6c43788ae9c702d. Latest consensus state: current_term: 1 leader_uuid: "da0ad6333fd24309a6c43788ae9c702d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } } }
I20260812 16:32:16.394410 22932 tablet_replica.cc:442] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:32:16.394747 22935 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 16:32:16.394855 22935 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 16:32:16.394827 22344 catalog_manager.cc:5495] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d reported cstate change: term changed from 0 to 1, leader changed from <none> to da0ad6333fd24309a6c43788ae9c702d (127.19.238.66). New cstate: current_term: 1 leader_uuid: "da0ad6333fd24309a6c43788ae9c702d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } health_report { overall_health: UNKNOWN } } }
I20260812 16:32:16.405503 22927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.402691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:41375 (local address 127.0.0.1:54100)
0812 16:32:16.402806 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:16.402821 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:16.402884 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:16.403070 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:16.403071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:16.403073 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:32:16.403126 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.403129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.403814 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.403816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.404482 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.404485 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.404903 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.404905 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.404978 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.404994 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:32:16.405309 (+ 315us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:32:16.405321 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 16:32:16.405374 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 16:32:16.405426 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 16:32:16.405503 22905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.402796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54100 (local address 127.19.238.66:41375)
0812 16:32:16.402848 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:16.402850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:16.402863 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:32:16.402960 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:16.402961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:16.402981 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:16.403047 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:16.403198 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.403702 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.404590 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.404798 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.404881 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.405116 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:16.405224 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:16.405313 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:16.405380 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 16:32:16.405403 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 16:32:16.407635 22522 raft_consensus.cc:1261] T 885fc40ec21b49b592b14df7fd226cbf P 62a0236c035643e4a8686166beec2b86 [term 1 FOLLOWER]: Refusing update from remote peer da0ad6333fd24309a6c43788ae9c702d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:16.408015 22833 raft_consensus.cc:1261] T 885fc40ec21b49b592b14df7fd226cbf P d7a59ddbded945de918ffa6937ea91e3 [term 1 FOLLOWER]: Refusing update from remote peer da0ad6333fd24309a6c43788ae9c702d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:16.408001 22915 consensus_queue.cc:1035] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d [LEADER]: Connected to new peer: Peer: permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:16.408331 22915 consensus_queue.cc:1035] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:16.409336 22916 tablet_replica.cc:439] T 885fc40ec21b49b592b14df7fd226cbf P 62a0236c035643e4a8686166beec2b86: TxnStatusTablet state changed. Reason: New leader da0ad6333fd24309a6c43788ae9c702d. Latest consensus state: current_term: 1 leader_uuid: "da0ad6333fd24309a6c43788ae9c702d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } } }
I20260812 16:32:16.409437 22916 tablet_replica.cc:442] T 885fc40ec21b49b592b14df7fd226cbf P 62a0236c035643e4a8686166beec2b86: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:16.409592 22918 tablet_replica.cc:439] T 885fc40ec21b49b592b14df7fd226cbf P d7a59ddbded945de918ffa6937ea91e3: TxnStatusTablet state changed. Reason: New leader da0ad6333fd24309a6c43788ae9c702d. Latest consensus state: current_term: 1 leader_uuid: "da0ad6333fd24309a6c43788ae9c702d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } } }
I20260812 16:32:16.409708 22918 tablet_replica.cc:442] T 885fc40ec21b49b592b14df7fd226cbf P d7a59ddbded945de918ffa6937ea91e3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:16.410073 22932 tablet_replica.cc:439] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da0ad6333fd24309a6c43788ae9c702d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } } }
I20260812 16:32:16.410157 22932 tablet_replica.cc:442] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:32:16.410169 22916 tablet_replica.cc:439] T 885fc40ec21b49b592b14df7fd226cbf P 62a0236c035643e4a8686166beec2b86: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da0ad6333fd24309a6c43788ae9c702d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } } }
I20260812 16:32:16.410238 22916 tablet_replica.cc:442] T 885fc40ec21b49b592b14df7fd226cbf P 62a0236c035643e4a8686166beec2b86: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:16.410838 22939 mvcc.cc:205] Tried to move back new op lower bound from 7317718369922826240 to 7317718369869860864. Current Snapshot: MvccSnapshot[applied={T|T < 7317718369922826240}]
I20260812 16:32:16.410959 22915 tablet_replica.cc:439] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da0ad6333fd24309a6c43788ae9c702d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } } }
I20260812 16:32:16.411044 22915 tablet_replica.cc:442] T 885fc40ec21b49b592b14df7fd226cbf P da0ad6333fd24309a6c43788ae9c702d: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:32:16.411208 22918 tablet_replica.cc:439] T 885fc40ec21b49b592b14df7fd226cbf P d7a59ddbded945de918ffa6937ea91e3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da0ad6333fd24309a6c43788ae9c702d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } } }
I20260812 16:32:16.411286 22918 tablet_replica.cc:442] T 885fc40ec21b49b592b14df7fd226cbf P d7a59ddbded945de918ffa6937ea91e3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:16.411351 22941 mvcc.cc:205] Tried to move back new op lower bound from 7317718369922826240 to 7317718369869860864. Current Snapshot: MvccSnapshot[applied={T|T < 7317718369922826240}]
I20260812 16:32:16.411563 22938 mvcc.cc:205] Tried to move back new op lower bound from 7317718369922826240 to 7317718369869860864. Current Snapshot: MvccSnapshot[applied={T|T < 7317718369922826240}]
I20260812 16:32:16.416644 22909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.413586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55044 (local address 127.19.238.65:35523)
0812 16:32:16.413641 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:16.413643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:16.413652 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:32:16.413710 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:16.413711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:16.413726 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:16.413778 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:16.413933 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.414475 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.415672 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.416017 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.416090 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.416304 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:16.416432 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:16.416525 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:16.416544 (+ 19us) server_negotiation.cc:299] Negotiation successful
0812 16:32:16.416570 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 16:32:16.422011 22905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.419552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:46655 (local address 127.19.238.66:41375)
0812 16:32:16.419605 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:16.419607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:16.419666 (+ 59us) server_negotiation.cc:407] Connection header received
0812 16:32:16.419736 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:16.419737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:16.419747 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:16.419803 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:16.419987 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.420457 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.421246 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.421461 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.421548 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.421755 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:16.421788 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:16.421859 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:16.421920 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 16:32:16.421939 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 16:32:16.422047 22949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.419453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:41375 (local address 127.19.238.65:46655)
0812 16:32:16.419611 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:16.419623 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:16.419684 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:16.419813 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:16.419814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:16.419815 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:32:16.419904 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.419906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.420550 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.420551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.421156 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.421157 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.421552 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.421553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.421617 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.421639 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:32:16.421861 (+ 222us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:32:16.421863 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:32:16.421916 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 16:32:16.421960 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
I20260812 16:32:16.425040 22833 raft_consensus.cc:1261] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3 [term 1 FOLLOWER]: Refusing update from remote peer 62a0236c035643e4a8686166beec2b86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:16.425185 22676 raft_consensus.cc:1261] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d [term 1 FOLLOWER]: Refusing update from remote peer 62a0236c035643e4a8686166beec2b86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:16.425329 22920 consensus_queue.cc:1035] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:16.425546 22920 consensus_queue.cc:1035] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:16.435094 22909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.431872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:39015 (local address 127.19.238.65:35523)
0812 16:32:16.431961 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:16.431964 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:16.432045 (+ 81us) server_negotiation.cc:407] Connection header received
0812 16:32:16.432116 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:16.432117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:16.432128 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:16.432175 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:16.432430 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.433102 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.434278 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.434547 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.434604 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.434826 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:16.434864 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:16.434893 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:16.434999 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 16:32:16.435019 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 16:32:16.435127 22955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.431810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:35523 (local address 127.19.238.66:39015)
0812 16:32:16.431983 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:16.431995 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:16.432065 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:16.432200 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:16.432201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:16.432203 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:32:16.432327 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.432329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.433193 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.433194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.434192 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.434195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.434629 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.434630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.434700 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.434717 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:32:16.434935 (+ 218us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:32:16.434936 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 16:32:16.434982 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 16:32:16.435025 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260812 16:32:16.454599 22344 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:16.454741 22344 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:16.455729 22344 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f: Sending DeleteTablet for 3 replicas of tablet 0739c167345645aaae7ffb7ad1fdb3a1
I20260812 16:32:16.456224 22653 tablet_service.cc:1514] Processing DeleteTablet for tablet 0739c167345645aaae7ffb7ad1fdb3a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:16 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:49518
I20260812 16:32:16.456396 22812 tablet_service.cc:1514] Processing DeleteTablet for tablet 0739c167345645aaae7ffb7ad1fdb3a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:16 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:46210
I20260812 16:32:16.456755 22344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:32:16.456913 22344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:32:16.457041 22498 tablet_service.cc:1514] Processing DeleteTablet for tablet 0739c167345645aaae7ffb7ad1fdb3a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:16 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:33156
I20260812 16:32:16.457324 22959 tablet_replica.cc:330] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3: stopping tablet replica
I20260812 16:32:16.457446 22959 raft_consensus.cc:2229] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:16.457517 22959 raft_consensus.cc:2258] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:16.457685 22958 tablet_replica.cc:330] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d: stopping tablet replica
I20260812 16:32:16.457789 22958 raft_consensus.cc:2229] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:16.457834 22958 raft_consensus.cc:2258] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:16.458326 22958 ts_tablet_manager.cc:1905] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:16.458802 22653 tablet_service.cc:1467] Processing CreateTablet for tablet 47b88ba7c87d412eb0f97311246446b6 (DEFAULT_TABLE table=test-table [id=6f10768425d849fd8e186d9e6f2d6f63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:16.458940 22653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47b88ba7c87d412eb0f97311246446b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:16.460203 22958 ts_tablet_manager.cc:1918] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:32:16.460276 22958 log.cc:1196] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/0739c167345645aaae7ffb7ad1fdb3a1
I20260812 16:32:16.460350 22911 tablet_bootstrap.cc:492] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d: Bootstrap starting.
I20260812 16:32:16.460567 22958 ts_tablet_manager.cc:1939] T 0739c167345645aaae7ffb7ad1fdb3a1 P da0ad6333fd24309a6c43788ae9c702d: Deleting consensus metadata
I20260812 16:32:16.461006 22911 tablet_bootstrap.cc:654] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:16.461216 22812 tablet_service.cc:1467] Processing CreateTablet for tablet 47b88ba7c87d412eb0f97311246446b6 (DEFAULT_TABLE table=test-table [id=6f10768425d849fd8e186d9e6f2d6f63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:16.461347 22812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47b88ba7c87d412eb0f97311246446b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:16.461699 22328 catalog_manager.cc:4841] TS da0ad6333fd24309a6c43788ae9c702d (127.19.238.66:41375): tablet 0739c167345645aaae7ffb7ad1fdb3a1 (table test-table [id=fb00e4e38e014350aaa96a49e429f986]) successfully deleted
I20260812 16:32:16.462805 22914 tablet_bootstrap.cc:492] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3: Bootstrap starting.
I20260812 16:32:16.463224 22914 tablet_bootstrap.cc:654] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:16.463541 22498 tablet_service.cc:1467] Processing CreateTablet for tablet 47b88ba7c87d412eb0f97311246446b6 (DEFAULT_TABLE table=test-table [id=6f10768425d849fd8e186d9e6f2d6f63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:16.463696 22498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47b88ba7c87d412eb0f97311246446b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:16.464039 22914 tablet_bootstrap.cc:492] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3: No bootstrap required, opened a new log
I20260812 16:32:16.464090 22914 ts_tablet_manager.cc:1397] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:16.464233 22914 raft_consensus.cc:348] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:16.464311 22914 raft_consensus.cc:374] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:16.464331 22914 raft_consensus.cc:729] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a59ddbded945de918ffa6937ea91e3, State: Initialized, Role: FOLLOWER
I20260812 16:32:16.464373 22914 consensus_queue.cc:260] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:16.464450 22914 ts_tablet_manager.cc:1428] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:16.465152 22910 tablet_bootstrap.cc:492] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86: Bootstrap starting.
I20260812 16:32:16.465629 22910 tablet_bootstrap.cc:654] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:16.465721 22959 ts_tablet_manager.cc:1905] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:16.466358 22910 tablet_bootstrap.cc:492] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86: No bootstrap required, opened a new log
I20260812 16:32:16.466408 22910 ts_tablet_manager.cc:1397] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:16.466553 22910 raft_consensus.cc:348] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:16.466620 22910 raft_consensus.cc:374] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:16.466639 22910 raft_consensus.cc:729] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a0236c035643e4a8686166beec2b86, State: Initialized, Role: FOLLOWER
I20260812 16:32:16.466681 22910 consensus_queue.cc:260] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:16.466777 22910 ts_tablet_manager.cc:1428] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:16.467785 22959 ts_tablet_manager.cc:1918] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:32:16.467895 22959 log.cc:1196] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/0739c167345645aaae7ffb7ad1fdb3a1
I20260812 16:32:16.468182 22959 ts_tablet_manager.cc:1939] T 0739c167345645aaae7ffb7ad1fdb3a1 P d7a59ddbded945de918ffa6937ea91e3: Deleting consensus metadata
I20260812 16:32:16.468644 22327 catalog_manager.cc:4841] TS d7a59ddbded945de918ffa6937ea91e3 (127.19.238.67:44449): tablet 0739c167345645aaae7ffb7ad1fdb3a1 (table test-table [id=fb00e4e38e014350aaa96a49e429f986]) successfully deleted
I20260812 16:32:16.468792 22960 tablet_replica.cc:330] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86: stopping tablet replica
I20260812 16:32:16.468919 22960 raft_consensus.cc:2229] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:32:16.469028 22960 raft_consensus.cc:2258] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:16.469626 22960 ts_tablet_manager.cc:1905] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:16.471623 22960 ts_tablet_manager.cc:1918] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:32:16.471706 22960 log.cc:1196] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/wal/wals/0739c167345645aaae7ffb7ad1fdb3a1
I20260812 16:32:16.472030 22960 ts_tablet_manager.cc:1939] T 0739c167345645aaae7ffb7ad1fdb3a1 P 62a0236c035643e4a8686166beec2b86: Deleting consensus metadata
I20260812 16:32:16.472385 22911 tablet_bootstrap.cc:492] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d: No bootstrap required, opened a new log
I20260812 16:32:16.472448 22911 ts_tablet_manager.cc:1397] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20260812 16:32:16.472479 22328 catalog_manager.cc:4841] TS 62a0236c035643e4a8686166beec2b86 (127.19.238.65:35523): tablet 0739c167345645aaae7ffb7ad1fdb3a1 (table test-table [id=fb00e4e38e014350aaa96a49e429f986]) successfully deleted
I20260812 16:32:16.472579 22911 raft_consensus.cc:348] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:16.472695 22911 raft_consensus.cc:374] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:16.472718 22911 raft_consensus.cc:729] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da0ad6333fd24309a6c43788ae9c702d, State: Initialized, Role: FOLLOWER
I20260812 16:32:16.472769 22911 consensus_queue.cc:260] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:16.472867 22911 ts_tablet_manager.cc:1428] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:16.479944 22915 raft_consensus.cc:482] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:32:16.480046 22915 raft_consensus.cc:504] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:16.480211 22915 leader_election.cc:288] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62a0236c035643e4a8686166beec2b86 (127.19.238.65:35523), d7a59ddbded945de918ffa6937ea91e3 (127.19.238.67:44449)
I20260812 16:32:16.480403 22833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47b88ba7c87d412eb0f97311246446b6" candidate_uuid: "da0ad6333fd24309a6c43788ae9c702d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a59ddbded945de918ffa6937ea91e3" is_pre_election: true
I20260812 16:32:16.480438 22522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47b88ba7c87d412eb0f97311246446b6" candidate_uuid: "da0ad6333fd24309a6c43788ae9c702d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a0236c035643e4a8686166beec2b86" is_pre_election: true
I20260812 16:32:16.480521 22833 raft_consensus.cc:2454] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da0ad6333fd24309a6c43788ae9c702d in term 0.
I20260812 16:32:16.480526 22522 raft_consensus.cc:2454] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da0ad6333fd24309a6c43788ae9c702d in term 0.
I20260812 16:32:16.480662 22599 leader_election.cc:302] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a0236c035643e4a8686166beec2b86, da0ad6333fd24309a6c43788ae9c702d; no voters:
I20260812 16:32:16.480754 22915 raft_consensus.cc:2781] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:32:16.480787 22915 raft_consensus.cc:482] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:32:16.480808 22915 raft_consensus.cc:3037] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:16.481618 22915 raft_consensus.cc:504] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:16.481762 22915 leader_election.cc:288] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [CANDIDATE]: Term 1 election: Requested vote from peers 62a0236c035643e4a8686166beec2b86 (127.19.238.65:35523), d7a59ddbded945de918ffa6937ea91e3 (127.19.238.67:44449)
I20260812 16:32:16.481956 22833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47b88ba7c87d412eb0f97311246446b6" candidate_uuid: "da0ad6333fd24309a6c43788ae9c702d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a59ddbded945de918ffa6937ea91e3"
I20260812 16:32:16.482000 22522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47b88ba7c87d412eb0f97311246446b6" candidate_uuid: "da0ad6333fd24309a6c43788ae9c702d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a0236c035643e4a8686166beec2b86"
I20260812 16:32:16.482036 22833 raft_consensus.cc:3037] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:16.482067 22522 raft_consensus.cc:3037] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:16.482841 22522 raft_consensus.cc:2454] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da0ad6333fd24309a6c43788ae9c702d in term 1.
I20260812 16:32:16.482846 22833 raft_consensus.cc:2454] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da0ad6333fd24309a6c43788ae9c702d in term 1.
I20260812 16:32:16.482997 22599 leader_election.cc:302] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a0236c035643e4a8686166beec2b86, da0ad6333fd24309a6c43788ae9c702d; no voters:
I20260812 16:32:16.483083 22915 raft_consensus.cc:2781] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:16.483126 22915 raft_consensus.cc:686] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [term 1 LEADER]: Becoming Leader. State: Replica: da0ad6333fd24309a6c43788ae9c702d, State: Running, Role: LEADER
I20260812 16:32:16.483170 22915 consensus_queue.cc:237] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:16.483752 22343 catalog_manager.cc:5495] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d reported cstate change: term changed from 0 to 1, leader changed from <none> to da0ad6333fd24309a6c43788ae9c702d (127.19.238.66). New cstate: current_term: 1 leader_uuid: "da0ad6333fd24309a6c43788ae9c702d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } health_report { overall_health: UNKNOWN } } }
I20260812 16:32:16.497377 22905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.494347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54106 (local address 127.19.238.66:41375)
0812 16:32:16.494386 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:16.494388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:16.494397 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:32:16.494475 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:16.494476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:16.494489 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:16.494554 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:16.494700 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.495436 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.496561 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.496862 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.496936 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.497108 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:16.497144 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:16.497199 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:16.497273 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 16:32:16.497296 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:32:16.502818 22905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.499851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:51253 (local address 127.19.238.66:41375)
0812 16:32:16.499885 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:16.499887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:16.499926 (+ 39us) server_negotiation.cc:407] Connection header received
0812 16:32:16.499994 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:16.499995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:16.500003 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:16.500059 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:16.500179 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.500951 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.501990 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.502283 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.502356 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.502533 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:16.502564 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:16.502619 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:16.502693 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 16:32:16.502714 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 16:32:16.502907 22955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:16.499762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:41375 (local address 127.19.238.66:51253)
0812 16:32:16.499858 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:16.499884 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:16.499943 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:16.500069 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:16.500069 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:16.500070 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:32:16.500112 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.500113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.501044 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.501045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.501894 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:16.501897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:16.502376 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:16.502377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:16.502441 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:16.502447 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:32:16.502638 (+ 191us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:32:16.502641 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:32:16.502693 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 16:32:16.502740 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 16:32:16.505632 22522 raft_consensus.cc:1261] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86 [term 1 FOLLOWER]: Refusing update from remote peer da0ad6333fd24309a6c43788ae9c702d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:16.505669 22833 raft_consensus.cc:1261] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3 [term 1 FOLLOWER]: Refusing update from remote peer da0ad6333fd24309a6c43788ae9c702d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:16.505857 22932 consensus_queue.cc:1035] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:16.505980 22932 consensus_queue.cc:1035] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [LEADER]: Connected to new peer: Peer: permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 16:32:16.512784 22938 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 16:32:18.041631 22344 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:18.041757 22344 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:18.043010 22344 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f: Sending DeleteTablet for 3 replicas of tablet 47b88ba7c87d412eb0f97311246446b6
I20260812 16:32:18.043370 22498 tablet_service.cc:1514] Processing DeleteTablet for tablet 47b88ba7c87d412eb0f97311246446b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:18 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:33156
I20260812 16:32:18.043501 22812 tablet_service.cc:1514] Processing DeleteTablet for tablet 47b88ba7c87d412eb0f97311246446b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:18 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:46210
I20260812 16:32:18.043501 22653 tablet_service.cc:1514] Processing DeleteTablet for tablet 47b88ba7c87d412eb0f97311246446b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:18 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:49518
I20260812 16:32:18.043677 22993 tablet_replica.cc:330] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86: stopping tablet replica
I20260812 16:32:18.043752 22993 raft_consensus.cc:2229] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:18.043802 22993 raft_consensus.cc:2258] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:18.044282 22993 ts_tablet_manager.cc:1905] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:18.046559 22993 ts_tablet_manager.cc:1918] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 16:32:18.046650 22993 log.cc:1196] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/wal/wals/47b88ba7c87d412eb0f97311246446b6
I20260812 16:32:18.046896 22993 ts_tablet_manager.cc:1939] T 47b88ba7c87d412eb0f97311246446b6 P 62a0236c035643e4a8686166beec2b86: Deleting consensus metadata
I20260812 16:32:18.043736 22995 tablet_replica.cc:330] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d: stopping tablet replica
I20260812 16:32:18.047324 22328 catalog_manager.cc:4841] TS 62a0236c035643e4a8686166beec2b86 (127.19.238.65:35523): tablet 47b88ba7c87d412eb0f97311246446b6 (table test-table [id=6f10768425d849fd8e186d9e6f2d6f63]) successfully deleted
I20260812 16:32:18.047377 22995 raft_consensus.cc:2229] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:32:18.047501 22995 raft_consensus.cc:2258] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:18.047780 22995 ts_tablet_manager.cc:1905] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:18.049394 22995 ts_tablet_manager.cc:1918] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 16:32:18.049470 22995 log.cc:1196] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/47b88ba7c87d412eb0f97311246446b6
I20260812 16:32:18.049764 22995 ts_tablet_manager.cc:1939] T 47b88ba7c87d412eb0f97311246446b6 P da0ad6333fd24309a6c43788ae9c702d: Deleting consensus metadata
I20260812 16:32:18.050344 22328 catalog_manager.cc:4841] TS da0ad6333fd24309a6c43788ae9c702d (127.19.238.66:41375): tablet 47b88ba7c87d412eb0f97311246446b6 (table test-table [id=6f10768425d849fd8e186d9e6f2d6f63]) successfully deleted
I20260812 16:32:18.050580 22994 tablet_replica.cc:330] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3: stopping tablet replica
I20260812 16:32:18.050668 22994 raft_consensus.cc:2229] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:18.050729 22994 raft_consensus.cc:2258] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:18.051004 22994 ts_tablet_manager.cc:1905] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:18.053016 22994 ts_tablet_manager.cc:1918] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 16:32:18.053100 22994 log.cc:1196] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/47b88ba7c87d412eb0f97311246446b6
I20260812 16:32:18.053321 22994 ts_tablet_manager.cc:1939] T 47b88ba7c87d412eb0f97311246446b6 P d7a59ddbded945de918ffa6937ea91e3: Deleting consensus metadata
I20260812 16:32:18.053696 22327 catalog_manager.cc:4841] TS d7a59ddbded945de918ffa6937ea91e3 (127.19.238.67:44449): tablet 47b88ba7c87d412eb0f97311246446b6 (table test-table [id=6f10768425d849fd8e186d9e6f2d6f63]) successfully deleted
Aug 12 16:32:18 dist-test-slave-mvvj krb5kdc[22292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552338, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 16:32:18 dist-test-slave-mvvj krb5kdc[22292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552338, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:18.077003 23004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:18.064150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38564 (local address 127.19.238.126:40441)
0812 16:32:18.064358 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:18.064360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:18.064371 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:32:18.064488 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:18.064489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:18.064502 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:18.064594 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:18.065261 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:18.065799 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:18.066573 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:18.066790 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:18.066864 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:18.074329 (+ 7465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:18.074332 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:18.074332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:18.074343 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:18.075773 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:18.076244 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:18.076246 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:18.076246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:18.076308 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:18.076485 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:18.076486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:18.076486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:18.076613 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:18.076693 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:18.076808 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 16:32:18.076922 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":122,"threads_started":1}
I20260812 16:32:18.078778 22344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:32:18.079010 22344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:32:18.080745 22498 tablet_service.cc:1467] Processing CreateTablet for tablet 9b447c0bc5bf4b4e8a123e35c77db56a (DEFAULT_TABLE table=test-table [id=dae15a8a33b54b29947a09d9256d3943]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:18.080742 22812 tablet_service.cc:1467] Processing CreateTablet for tablet 9b447c0bc5bf4b4e8a123e35c77db56a (DEFAULT_TABLE table=test-table [id=dae15a8a33b54b29947a09d9256d3943]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:18.080742 22653 tablet_service.cc:1467] Processing CreateTablet for tablet 9b447c0bc5bf4b4e8a123e35c77db56a (DEFAULT_TABLE table=test-table [id=dae15a8a33b54b29947a09d9256d3943]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:18.080893 22812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b447c0bc5bf4b4e8a123e35c77db56a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:18.080895 22653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b447c0bc5bf4b4e8a123e35c77db56a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:18.080906 22498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b447c0bc5bf4b4e8a123e35c77db56a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:18.082221 23005 tablet_bootstrap.cc:492] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86: Bootstrap starting.
I20260812 16:32:18.082646 23005 tablet_bootstrap.cc:654] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:18.082998 23007 tablet_bootstrap.cc:492] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3: Bootstrap starting.
I20260812 16:32:18.083392 23005 tablet_bootstrap.cc:492] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86: No bootstrap required, opened a new log
I20260812 16:32:18.083446 23007 tablet_bootstrap.cc:654] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:18.083458 23005 ts_tablet_manager.cc:1397] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:18.083598 23005 raft_consensus.cc:348] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:18.083698 23005 raft_consensus.cc:374] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:18.083755 23005 raft_consensus.cc:729] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a0236c035643e4a8686166beec2b86, State: Initialized, Role: FOLLOWER
I20260812 16:32:18.083810 23008 tablet_bootstrap.cc:492] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d: Bootstrap starting.
I20260812 16:32:18.083807 23005 consensus_queue.cc:260] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:18.084049 23005 ts_tablet_manager.cc:1428] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:18.084254 23008 tablet_bootstrap.cc:654] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:18.084770 23007 tablet_bootstrap.cc:492] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3: No bootstrap required, opened a new log
I20260812 16:32:18.084827 23007 ts_tablet_manager.cc:1397] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:32:18.084991 23007 raft_consensus.cc:348] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:18.085075 23007 raft_consensus.cc:374] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:18.085100 23007 raft_consensus.cc:729] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a59ddbded945de918ffa6937ea91e3, State: Initialized, Role: FOLLOWER
I20260812 16:32:18.085155 23007 consensus_queue.cc:260] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:18.085326 23007 ts_tablet_manager.cc:1428] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:18.085628 23008 tablet_bootstrap.cc:492] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d: No bootstrap required, opened a new log
I20260812 16:32:18.085686 23008 ts_tablet_manager.cc:1397] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 16:32:18.085810 23008 raft_consensus.cc:348] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:18.085872 23008 raft_consensus.cc:374] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:18.085887 23008 raft_consensus.cc:729] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da0ad6333fd24309a6c43788ae9c702d, State: Initialized, Role: FOLLOWER
I20260812 16:32:18.085985 23008 consensus_queue.cc:260] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:18.086093 23008 ts_tablet_manager.cc:1428] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 16:32:18.167368 23011 raft_consensus.cc:482] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:32:18.167485 23011 raft_consensus.cc:504] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:18.167685 23011 leader_election.cc:288] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da0ad6333fd24309a6c43788ae9c702d (127.19.238.66:41375), d7a59ddbded945de918ffa6937ea91e3 (127.19.238.67:44449)
I20260812 16:32:18.167968 22676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b447c0bc5bf4b4e8a123e35c77db56a" candidate_uuid: "62a0236c035643e4a8686166beec2b86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da0ad6333fd24309a6c43788ae9c702d" is_pre_election: true
I20260812 16:32:18.167982 22833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b447c0bc5bf4b4e8a123e35c77db56a" candidate_uuid: "62a0236c035643e4a8686166beec2b86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a59ddbded945de918ffa6937ea91e3" is_pre_election: true
I20260812 16:32:18.168088 22833 raft_consensus.cc:2454] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a0236c035643e4a8686166beec2b86 in term 0.
I20260812 16:32:18.168092 22676 raft_consensus.cc:2454] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a0236c035643e4a8686166beec2b86 in term 0.
I20260812 16:32:18.168325 22442 leader_election.cc:302] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a0236c035643e4a8686166beec2b86, d7a59ddbded945de918ffa6937ea91e3; no voters:
I20260812 16:32:18.168470 23011 raft_consensus.cc:2781] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:32:18.168514 23011 raft_consensus.cc:482] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:32:18.168534 23011 raft_consensus.cc:3037] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:18.169356 23011 raft_consensus.cc:504] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:18.169505 23011 leader_election.cc:288] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [CANDIDATE]: Term 1 election: Requested vote from peers da0ad6333fd24309a6c43788ae9c702d (127.19.238.66:41375), d7a59ddbded945de918ffa6937ea91e3 (127.19.238.67:44449)
I20260812 16:32:18.169677 22676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b447c0bc5bf4b4e8a123e35c77db56a" candidate_uuid: "62a0236c035643e4a8686166beec2b86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da0ad6333fd24309a6c43788ae9c702d"
I20260812 16:32:18.169757 22676 raft_consensus.cc:3037] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:18.169727 22833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b447c0bc5bf4b4e8a123e35c77db56a" candidate_uuid: "62a0236c035643e4a8686166beec2b86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a59ddbded945de918ffa6937ea91e3"
I20260812 16:32:18.169839 22833 raft_consensus.cc:3037] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:18.170408 22676 raft_consensus.cc:2454] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a0236c035643e4a8686166beec2b86 in term 1.
I20260812 16:32:18.170550 22833 raft_consensus.cc:2454] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a0236c035643e4a8686166beec2b86 in term 1.
I20260812 16:32:18.170595 22443 leader_election.cc:302] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a0236c035643e4a8686166beec2b86, da0ad6333fd24309a6c43788ae9c702d; no voters:
I20260812 16:32:18.170702 23011 raft_consensus.cc:2781] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:18.170897 23011 raft_consensus.cc:686] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [term 1 LEADER]: Becoming Leader. State: Replica: 62a0236c035643e4a8686166beec2b86, State: Running, Role: LEADER
I20260812 16:32:18.170969 23011 consensus_queue.cc:237] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:18.171583 22344 catalog_manager.cc:5495] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62a0236c035643e4a8686166beec2b86 (127.19.238.65). New cstate: current_term: 1 leader_uuid: "62a0236c035643e4a8686166beec2b86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } health_report { overall_health: UNKNOWN } } }
I20260812 16:32:18.181555 23017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:18.179229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55056 (local address 127.19.238.65:35523)
0812 16:32:18.179376 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:18.179378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:18.179384 (+ 6us) server_negotiation.cc:407] Connection header received
0812 16:32:18.179402 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:18.179403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:18.179413 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:18.179462 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:18.179600 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:18.180158 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:18.180938 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:18.181142 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:18.181197 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:18.181325 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:18.181351 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:18.181389 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:18.181426 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 16:32:18.181444 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":94,"threads_started":1}
I20260812 16:32:18.184026 22676 raft_consensus.cc:1261] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d [term 1 FOLLOWER]: Refusing update from remote peer 62a0236c035643e4a8686166beec2b86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:18.184161 22833 raft_consensus.cc:1261] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3 [term 1 FOLLOWER]: Refusing update from remote peer 62a0236c035643e4a8686166beec2b86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:18.184270 23014 consensus_queue.cc:1035] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:18.184402 23014 consensus_queue.cc:1035] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:18.194383 22343 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:18.194487 22343 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:18.195037 22343 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f: Sending DeleteTablet for 3 replicas of tablet 9b447c0bc5bf4b4e8a123e35c77db56a
I20260812 16:32:18.195276 22498 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b447c0bc5bf4b4e8a123e35c77db56a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:18 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:33156
I20260812 16:32:18.195389 22812 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b447c0bc5bf4b4e8a123e35c77db56a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:18 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:46210
I20260812 16:32:18.195407 22993 tablet_replica.cc:330] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86: stopping tablet replica
I20260812 16:32:18.195478 22994 tablet_replica.cc:330] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3: stopping tablet replica
I20260812 16:32:18.195483 22993 raft_consensus.cc:2229] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:32:18.195529 22994 raft_consensus.cc:2229] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:18.195546 22993 raft_consensus.cc:2258] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:18.195581 22994 raft_consensus.cc:2258] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:18.195524 22343 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:32:18.195663 22343 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:32:18.195907 22653 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b447c0bc5bf4b4e8a123e35c77db56a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:18 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:49518
I20260812 16:32:18.196007 22994 ts_tablet_manager.cc:1905] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:18.196010 22995 tablet_replica.cc:330] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d: stopping tablet replica
I20260812 16:32:18.196074 22995 raft_consensus.cc:2229] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:18.196112 22995 raft_consensus.cc:2258] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:18.196190 22993 ts_tablet_manager.cc:1905] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:18.196990 22995 ts_tablet_manager.cc:1905] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:18.197014 22812 tablet_service.cc:1467] Processing CreateTablet for tablet df3e60196aa54b5aa85253aa89733f99 (DEFAULT_TABLE table=test-table [id=bca9822131524805bc3b1f6b2c7cde77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:18.197120 22812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df3e60196aa54b5aa85253aa89733f99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:18.197803 22994 ts_tablet_manager.cc:1918] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:32:18.197803 22993 ts_tablet_manager.cc:1918] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:32:18.197855 22993 log.cc:1196] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-0/wal/wals/9b447c0bc5bf4b4e8a123e35c77db56a
I20260812 16:32:18.197855 22994 log.cc:1196] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/9b447c0bc5bf4b4e8a123e35c77db56a
I20260812 16:32:18.198092 22994 ts_tablet_manager.cc:1939] T 9b447c0bc5bf4b4e8a123e35c77db56a P d7a59ddbded945de918ffa6937ea91e3: Deleting consensus metadata
I20260812 16:32:18.198112 22993 ts_tablet_manager.cc:1939] T 9b447c0bc5bf4b4e8a123e35c77db56a P 62a0236c035643e4a8686166beec2b86: Deleting consensus metadata
I20260812 16:32:18.198383 22653 tablet_service.cc:1467] Processing CreateTablet for tablet df3e60196aa54b5aa85253aa89733f99 (DEFAULT_TABLE table=test-table [id=bca9822131524805bc3b1f6b2c7cde77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:18.198500 22653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df3e60196aa54b5aa85253aa89733f99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:18.198719 22995 ts_tablet_manager.cc:1918] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:32:18.198794 22995 log.cc:1196] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/9b447c0bc5bf4b4e8a123e35c77db56a
I20260812 16:32:18.199060 22995 ts_tablet_manager.cc:1939] T 9b447c0bc5bf4b4e8a123e35c77db56a P da0ad6333fd24309a6c43788ae9c702d: Deleting consensus metadata
I20260812 16:32:18.199186 22327 catalog_manager.cc:4841] TS d7a59ddbded945de918ffa6937ea91e3 (127.19.238.67:44449): tablet 9b447c0bc5bf4b4e8a123e35c77db56a (table test-table [id=dae15a8a33b54b29947a09d9256d3943]) successfully deleted
I20260812 16:32:18.199317 22498 tablet_service.cc:1467] Processing CreateTablet for tablet df3e60196aa54b5aa85253aa89733f99 (DEFAULT_TABLE table=test-table [id=bca9822131524805bc3b1f6b2c7cde77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:18.199445 22498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df3e60196aa54b5aa85253aa89733f99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:18.199642 23008 tablet_bootstrap.cc:492] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d: Bootstrap starting.
I20260812 16:32:18.199673 23007 tablet_bootstrap.cc:492] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3: Bootstrap starting.
I20260812 16:32:18.200021 23008 tablet_bootstrap.cc:654] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:18.200044 23007 tablet_bootstrap.cc:654] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:18.200642 23007 tablet_bootstrap.cc:492] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3: No bootstrap required, opened a new log
I20260812 16:32:18.200665 22328 catalog_manager.cc:4841] TS 62a0236c035643e4a8686166beec2b86 (127.19.238.65:35523): tablet 9b447c0bc5bf4b4e8a123e35c77db56a (table test-table [id=dae15a8a33b54b29947a09d9256d3943]) successfully deleted
I20260812 16:32:18.200693 23007 ts_tablet_manager.cc:1397] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:18.200814 22328 catalog_manager.cc:4841] TS da0ad6333fd24309a6c43788ae9c702d (127.19.238.66:41375): tablet 9b447c0bc5bf4b4e8a123e35c77db56a (table test-table [id=dae15a8a33b54b29947a09d9256d3943]) successfully deleted
I20260812 16:32:18.200892 23005 tablet_bootstrap.cc:492] T df3e60196aa54b5aa85253aa89733f99 P 62a0236c035643e4a8686166beec2b86: Bootstrap starting.
I20260812 16:32:18.200855 23007 raft_consensus.cc:348] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:18.200937 23007 raft_consensus.cc:374] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:18.200963 23007 raft_consensus.cc:729] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a59ddbded945de918ffa6937ea91e3, State: Initialized, Role: FOLLOWER
I20260812 16:32:18.201012 23007 consensus_queue.cc:260] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:18.201115 23007 ts_tablet_manager.cc:1428] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:18.201252 23005 tablet_bootstrap.cc:654] T df3e60196aa54b5aa85253aa89733f99 P 62a0236c035643e4a8686166beec2b86: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:18.201495 23008 tablet_bootstrap.cc:492] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d: No bootstrap required, opened a new log
I20260812 16:32:18.201545 23008 ts_tablet_manager.cc:1397] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:32:18.201658 23008 raft_consensus.cc:348] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:18.201742 23008 raft_consensus.cc:374] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:18.201768 23008 raft_consensus.cc:729] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da0ad6333fd24309a6c43788ae9c702d, State: Initialized, Role: FOLLOWER
I20260812 16:32:18.201821 23008 consensus_queue.cc:260] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:18.201917 23005 tablet_bootstrap.cc:492] T df3e60196aa54b5aa85253aa89733f99 P 62a0236c035643e4a8686166beec2b86: No bootstrap required, opened a new log
I20260812 16:32:18.201938 23008 ts_tablet_manager.cc:1428] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:18.201967 23005 ts_tablet_manager.cc:1397] T df3e60196aa54b5aa85253aa89733f99 P 62a0236c035643e4a8686166beec2b86: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:18.202085 23005 raft_consensus.cc:348] T df3e60196aa54b5aa85253aa89733f99 P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:18.202162 23005 raft_consensus.cc:374] T df3e60196aa54b5aa85253aa89733f99 P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:18.202181 23005 raft_consensus.cc:729] T df3e60196aa54b5aa85253aa89733f99 P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a0236c035643e4a8686166beec2b86, State: Initialized, Role: FOLLOWER
I20260812 16:32:18.202252 23005 consensus_queue.cc:260] T df3e60196aa54b5aa85253aa89733f99 P 62a0236c035643e4a8686166beec2b86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:18.202349 23005 ts_tablet_manager.cc:1428] T df3e60196aa54b5aa85253aa89733f99 P 62a0236c035643e4a8686166beec2b86: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:18.299422 22976 raft_consensus.cc:482] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:32:18.299549 22976 raft_consensus.cc:504] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:18.299932 22976 leader_election.cc:288] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62a0236c035643e4a8686166beec2b86 (127.19.238.65:35523), d7a59ddbded945de918ffa6937ea91e3 (127.19.238.67:44449)
I20260812 16:32:18.300276 22833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df3e60196aa54b5aa85253aa89733f99" candidate_uuid: "da0ad6333fd24309a6c43788ae9c702d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a59ddbded945de918ffa6937ea91e3" is_pre_election: true
I20260812 16:32:18.300403 22833 raft_consensus.cc:2454] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da0ad6333fd24309a6c43788ae9c702d in term 0.
I20260812 16:32:18.300351 22522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df3e60196aa54b5aa85253aa89733f99" candidate_uuid: "da0ad6333fd24309a6c43788ae9c702d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a0236c035643e4a8686166beec2b86" is_pre_election: true
I20260812 16:32:18.300508 22522 raft_consensus.cc:2454] T df3e60196aa54b5aa85253aa89733f99 P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da0ad6333fd24309a6c43788ae9c702d in term 0.
I20260812 16:32:18.300638 22598 leader_election.cc:302] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7a59ddbded945de918ffa6937ea91e3, da0ad6333fd24309a6c43788ae9c702d; no voters:
I20260812 16:32:18.300740 22976 raft_consensus.cc:2781] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:32:18.300787 22976 raft_consensus.cc:482] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:32:18.300817 22976 raft_consensus.cc:3037] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:18.301584 22976 raft_consensus.cc:504] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:18.301741 22976 leader_election.cc:288] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [CANDIDATE]: Term 1 election: Requested vote from peers 62a0236c035643e4a8686166beec2b86 (127.19.238.65:35523), d7a59ddbded945de918ffa6937ea91e3 (127.19.238.67:44449)
I20260812 16:32:18.301904 22833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df3e60196aa54b5aa85253aa89733f99" candidate_uuid: "da0ad6333fd24309a6c43788ae9c702d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a59ddbded945de918ffa6937ea91e3"
I20260812 16:32:18.301954 22522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df3e60196aa54b5aa85253aa89733f99" candidate_uuid: "da0ad6333fd24309a6c43788ae9c702d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a0236c035643e4a8686166beec2b86"
I20260812 16:32:18.301996 22833 raft_consensus.cc:3037] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:18.302026 22522 raft_consensus.cc:3037] T df3e60196aa54b5aa85253aa89733f99 P 62a0236c035643e4a8686166beec2b86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:18.302776 22522 raft_consensus.cc:2454] T df3e60196aa54b5aa85253aa89733f99 P 62a0236c035643e4a8686166beec2b86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da0ad6333fd24309a6c43788ae9c702d in term 1.
I20260812 16:32:18.302783 22833 raft_consensus.cc:2454] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da0ad6333fd24309a6c43788ae9c702d in term 1.
I20260812 16:32:18.303014 22599 leader_election.cc:302] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 62a0236c035643e4a8686166beec2b86, d7a59ddbded945de918ffa6937ea91e3, da0ad6333fd24309a6c43788ae9c702d; no voters:
I20260812 16:32:18.303139 22976 raft_consensus.cc:2781] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:18.303207 22976 raft_consensus.cc:686] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [term 1 LEADER]: Becoming Leader. State: Replica: da0ad6333fd24309a6c43788ae9c702d, State: Running, Role: LEADER
I20260812 16:32:18.303268 22976 consensus_queue.cc:237] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } }
I20260812 16:32:18.303908 22344 catalog_manager.cc:5495] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d reported cstate change: term changed from 0 to 1, leader changed from <none> to da0ad6333fd24309a6c43788ae9c702d (127.19.238.66). New cstate: current_term: 1 leader_uuid: "da0ad6333fd24309a6c43788ae9c702d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da0ad6333fd24309a6c43788ae9c702d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 } health_report { overall_health: UNKNOWN } } }
I20260812 16:32:18.320461 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:18.318165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54114 (local address 127.19.238.66:41375)
0812 16:32:18.318335 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:18.318337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:18.318342 (+ 5us) server_negotiation.cc:407] Connection header received
0812 16:32:18.318360 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:18.318360 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:18.318371 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:18.318431 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:18.318608 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:18.319081 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:18.319798 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:18.320056 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:18.320111 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:18.320235 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:18.320261 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:18.320294 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:18.320325 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 16:32:18.320340 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":108,"threads_started":1}
I20260812 16:32:18.322726 22522 raft_consensus.cc:1261] T df3e60196aa54b5aa85253aa89733f99 P 62a0236c035643e4a8686166beec2b86 [term 1 FOLLOWER]: Refusing update from remote peer da0ad6333fd24309a6c43788ae9c702d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:18.322778 22833 raft_consensus.cc:1261] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3 [term 1 FOLLOWER]: Refusing update from remote peer da0ad6333fd24309a6c43788ae9c702d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:18.323091 22950 consensus_queue.cc:1035] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [LEADER]: Connected to new peer: Peer: permanent_uuid: "62a0236c035643e4a8686166beec2b86" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:18.323194 22976 consensus_queue.cc:1035] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7a59ddbded945de918ffa6937ea91e3" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 16:32:18.327929 22969 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 16:32:19.846364 22344 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:19.846696 22344 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:19.847771 22344 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1df569e0e14c4b24afd45ea55fac891f: Sending DeleteTablet for 3 replicas of tablet df3e60196aa54b5aa85253aa89733f99
I20260812 16:32:19.848119 22653 tablet_service.cc:1514] Processing DeleteTablet for tablet df3e60196aa54b5aa85253aa89733f99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:19 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:49518
I20260812 16:32:19.848142 22812 tablet_service.cc:1514] Processing DeleteTablet for tablet df3e60196aa54b5aa85253aa89733f99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:19 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:46210
I20260812 16:32:19.848137 22498 tablet_service.cc:1514] Processing DeleteTablet for tablet df3e60196aa54b5aa85253aa89733f99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:19 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:33156
I20260812 16:32:19.848228 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 22425
I20260812 16:32:19.853770 23053 tablet_replica.cc:330] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3: stopping tablet replica
I20260812 16:32:19.853894 23053 raft_consensus.cc:2229] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:19.853971 23053 raft_consensus.cc:2258] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:19.854113 23052 tablet_replica.cc:330] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d: stopping tablet replica
I20260812 16:32:19.854207 23052 raft_consensus.cc:2229] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:32:19.854292 23053 ts_tablet_manager.cc:1905] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:19.854321 23052 raft_consensus.cc:2258] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:19.854629 23052 ts_tablet_manager.cc:1905] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:19.856230 23053 ts_tablet_manager.cc:1918] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 16:32:19.856314 23053 log.cc:1196] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/df3e60196aa54b5aa85253aa89733f99
I20260812 16:32:19.856380 23052 ts_tablet_manager.cc:1918] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 16:32:19.856498 23052 log.cc:1196] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/df3e60196aa54b5aa85253aa89733f99
I20260812 16:32:19.856565 23053 ts_tablet_manager.cc:1939] T df3e60196aa54b5aa85253aa89733f99 P d7a59ddbded945de918ffa6937ea91e3: Deleting consensus metadata
I20260812 16:32:19.856793 23052 ts_tablet_manager.cc:1939] T df3e60196aa54b5aa85253aa89733f99 P da0ad6333fd24309a6c43788ae9c702d: Deleting consensus metadata
I20260812 16:32:19.857303 22327 catalog_manager.cc:4841] TS d7a59ddbded945de918ffa6937ea91e3 (127.19.238.67:44449): tablet df3e60196aa54b5aa85253aa89733f99 (table test-table [id=bca9822131524805bc3b1f6b2c7cde77]) successfully deleted
I20260812 16:32:19.857462 22328 catalog_manager.cc:4841] TS da0ad6333fd24309a6c43788ae9c702d (127.19.238.66:41375): tablet df3e60196aa54b5aa85253aa89733f99 (table test-table [id=bca9822131524805bc3b1f6b2c7cde77]) successfully deleted
W20260812 16:32:19.859311 22328 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.238.65:35523 (error 108)
I20260812 16:32:19.859593 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 22581
I20260812 16:32:19.861552 23057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:19.859761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:35523 (local address 127.0.0.1:55058)
0812 16:32:19.861450 (+ 1689us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:19.861508 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:35523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1660,"thread_start_us":89,"threads_started":1}
W20260812 16:32:19.861701 22328 catalog_manager.cc:4584] TS 62a0236c035643e4a8686166beec2b86 (127.19.238.65:35523): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df3e60196aa54b5aa85253aa89733f99: Network error: Client connection negotiation failed: client connection to 127.19.238.65:35523: connect: Connection refused (error 111)
I20260812 16:32:19.869366 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 22740
I20260812 16:32:19.877853 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 22308
2026-08-12T16:32:19Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5739 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:32:19 dist-test-slave-mvvj krb5kdc[23060](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:32:19 dist-test-slave-mvvj krb5kdc[23060](info): set up 2 sockets
Aug 12 16:32:19 dist-test-slave-mvvj krb5kdc[23060](info): commencing operation
krb5kdc: starting...
W20260812 16:32:21.027096 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:32:21 dist-test-slave-mvvj krb5kdc[23060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:32:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:32:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:32:21.164731 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:39811
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:34833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:39811
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:21.249260 23076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:21.249498 23076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:21.249524 23076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:21.250751 23076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:21.250797 23076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:21.250811 23076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:21.250824 23076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:21.250844 23076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:32:21.252439 23076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:34833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:39811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:39811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:21.252663 23076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:21.252857 23076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:21.257987 23081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:21.257972 23082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:21.258272 23076 server_base.cc:1034] running on GCE node
W20260812 16:32:21.257972 23085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:21.258621 23076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:21.258942 23076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:21.260128 23076 hybrid_clock.cc:648] HybridClock initialized: now 1786552341260107 us; error 37 us; skew 500 ppm
Aug 12 16:32:21 dist-test-slave-mvvj krb5kdc[23060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552341, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:21.262929 23076 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:32:21.263422 23076 webserver.cc:460] Webserver started at http://127.19.238.126:45931/ using document root <none> and password file <none>
I20260812 16:32:21.263638 23076 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:21.263681 23076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:21.263773 23076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:21.264938 23076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "46c65df6ae1e4ae3882f5c7d79687c2c"
format_stamp: "Formatted at 2026-08-12 16:32:21 on dist-test-slave-mvvj"
server_key: "46e38098fcf34584435141f0f94d18a7"
server_key_iv: "3c047e64d6d49f7d4399e6f6d455b37f"
server_key_version: "encryptionkey@0"
I20260812 16:32:21.265252 23076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "46c65df6ae1e4ae3882f5c7d79687c2c"
format_stamp: "Formatted at 2026-08-12 16:32:21 on dist-test-slave-mvvj"
server_key: "46e38098fcf34584435141f0f94d18a7"
server_key_iv: "3c047e64d6d49f7d4399e6f6d455b37f"
server_key_version: "encryptionkey@0"
I20260812 16:32:21.266681 23076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:32:21.267443 23093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:21.267668 23076 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:21.267760 23076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "46c65df6ae1e4ae3882f5c7d79687c2c"
format_stamp: "Formatted at 2026-08-12 16:32:21 on dist-test-slave-mvvj"
server_key: "46e38098fcf34584435141f0f94d18a7"
server_key_iv: "3c047e64d6d49f7d4399e6f6d455b37f"
server_key_version: "encryptionkey@0"
I20260812 16:32:21.267817 23076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:21.278533 23076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:21.280659 23076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:21.280776 23076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:21.284797 23076 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:39811
I20260812 16:32:21.284832 23159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:39811 every 8 connection(s)
I20260812 16:32:21.285200 23076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:32:21.286142 23160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:21.288720 23160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c: Bootstrap starting.
I20260812 16:32:21.289315 23160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:21.289618 23160 log.cc:824] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:21.290323 23160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c: No bootstrap required, opened a new log
I20260812 16:32:21.290900 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 23076
I20260812 16:32:21.291000 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:32:21.291175 20409 external_mini_cluster.cc:1428] Setting key 6cc9aab2d6d96fae697b6bdad367328d
I20260812 16:32:21.292613 23160 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46c65df6ae1e4ae3882f5c7d79687c2c" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39811 } }
I20260812 16:32:21.292814 23160 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:21.292866 23160 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c65df6ae1e4ae3882f5c7d79687c2c, State: Initialized, Role: FOLLOWER
I20260812 16:32:21.292989 23160 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46c65df6ae1e4ae3882f5c7d79687c2c" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39811 } }
I20260812 16:32:21.293071 23160 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:32:21.293093 23160 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:32:21.293143 23160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:21.294232 23160 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46c65df6ae1e4ae3882f5c7d79687c2c" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39811 } }
I20260812 16:32:21.294344 23160 leader_election.cc:302] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46c65df6ae1e4ae3882f5c7d79687c2c; no voters:
I20260812 16:32:21.294505 23160 leader_election.cc:288] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:32:21.294579 23165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:21.294757 23165 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c [term 1 LEADER]: Becoming Leader. State: Replica: 46c65df6ae1e4ae3882f5c7d79687c2c, State: Running, Role: LEADER
I20260812 16:32:21.294781 23160 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:21.294946 23165 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46c65df6ae1e4ae3882f5c7d79687c2c" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39811 } }
I20260812 16:32:21.296775 23166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46c65df6ae1e4ae3882f5c7d79687c2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46c65df6ae1e4ae3882f5c7d79687c2c" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39811 } } }
I20260812 16:32:21.296875 23166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:21.297082 23169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46c65df6ae1e4ae3882f5c7d79687c2c. Latest consensus state: current_term: 1 leader_uuid: "46c65df6ae1e4ae3882f5c7d79687c2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46c65df6ae1e4ae3882f5c7d79687c2c" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39811 } } }
I20260812 16:32:21.297154 23169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c [sys.catalog]: This master's current role is: LEADER
W20260812 16:32:21.297648 23182 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:32:21.297708 23182 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:32:21.297768 23183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:32:21.298002 23183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 16:32:21 dist-test-slave-mvvj krb5kdc[23060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:21.299759 23183 catalog_manager.cc:1347] Generated new cluster ID: 31c9b910d7dd480d86e543e342dc0545
I20260812 16:32:21.299814 23183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:32:21.303097 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:21.291727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33320 (local address 127.19.238.126:39811)
0812 16:32:21.291951 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:21.291961 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:21.291996 (+ 35us) server_negotiation.cc:407] Connection header received
0812 16:32:21.292576 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:21.292586 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:21.292653 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:21.292823 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:21.293715 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:21.295156 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:21.296415 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:21.297164 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:21.297297 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:21.299795 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:21.299802 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:21.299803 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:21.299813 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:21.301493 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:21.301873 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:21.301876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:21.301877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:21.301976 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:21.302191 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:21.302192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:21.302193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:21.302427 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:21.302623 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:21.302688 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 16:32:21.302829 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":115,"threads_started":1}
I20260812 16:32:21.330324 23183 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:32:21.331195 23183 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:21.349944 23183 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46c65df6ae1e4ae3882f5c7d79687c2c: Generated new TSK 0
I20260812 16:32:21.350205 23183 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
I20260812 16:32:21.383699 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:39811
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:34833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:21.478588 23188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:21.478837 23188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:21.478883 23188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:21.480110 23188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:21.480170 23188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:21.480216 23188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:32:21.481689 23188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:34833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:39811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:21.481958 23188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:21.482192 23188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:21.487414 23195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:21.487588 23188 server_base.cc:1034] running on GCE node
W20260812 16:32:21.487447 23194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:21.487432 23197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:21.487947 23188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:21.488160 23188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:21.489310 23188 hybrid_clock.cc:648] HybridClock initialized: now 1786552341489274 us; error 52 us; skew 500 ppm
Aug 12 16:32:21 dist-test-slave-mvvj krb5kdc[23060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552341, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:21.491998 23188 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:32:21.492506 23188 webserver.cc:460] Webserver started at http://127.19.238.65:45531/ using document root <none> and password file <none>
I20260812 16:32:21.492745 23188 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:21.492813 23188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:21.492975 23188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:21.494102 23188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "85eee359aed148849177ffffcd442c7f"
format_stamp: "Formatted at 2026-08-12 16:32:21 on dist-test-slave-mvvj"
server_key: "aae0185acd5b3401508096fbb0fab47e"
server_key_iv: "1b7be989f98539a3886049d3c7e5eb36"
server_key_version: "encryptionkey@0"
I20260812 16:32:21.494490 23188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "85eee359aed148849177ffffcd442c7f"
format_stamp: "Formatted at 2026-08-12 16:32:21 on dist-test-slave-mvvj"
server_key: "aae0185acd5b3401508096fbb0fab47e"
server_key_iv: "1b7be989f98539a3886049d3c7e5eb36"
server_key_version: "encryptionkey@0"
I20260812 16:32:21.496013 23188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:32:21.496932 23204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:21.497145 23188 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:21.497264 23188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "85eee359aed148849177ffffcd442c7f"
format_stamp: "Formatted at 2026-08-12 16:32:21 on dist-test-slave-mvvj"
server_key: "aae0185acd5b3401508096fbb0fab47e"
server_key_iv: "1b7be989f98539a3886049d3c7e5eb36"
server_key_version: "encryptionkey@0"
I20260812 16:32:21.497432 23188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:21.515029 23188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:21.517484 23188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:21.517719 23188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:21.517982 23188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:21.518334 23188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:21.518389 23188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:21.518433 23188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:21.518472 23188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:21.524449 23188 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:46311
I20260812 16:32:21.524560 23336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:46311 every 8 connection(s)
I20260812 16:32:21.524879 23188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:32:21.530704 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 23188
I20260812 16:32:21.530825 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:32:21.531037 20409 external_mini_cluster.cc:1428] Setting key 80ca3270e7711e2b7aaabcd19ad09e54
Aug 12 16:32:21 dist-test-slave-mvvj krb5kdc[23060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552341, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:21.536095 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:21.525658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:42713 (local address 127.19.238.126:39811)
0812 16:32:21.525738 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:21.525740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:21.525972 (+ 232us) server_negotiation.cc:407] Connection header received
0812 16:32:21.526552 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:21.526553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:21.526566 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:21.526626 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:21.527709 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:21.528360 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:21.529463 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:21.529713 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:21.529797 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:21.532888 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:21.532890 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:21.532890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:21.532899 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:21.534495 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:21.534973 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:21.534975 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:21.534976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:21.535040 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:21.535321 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:21.535323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:21.535323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:21.535457 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:21.535537 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:21.535892 (+ 355us) server_negotiation.cc:299] Negotiation successful
0812 16:32:21.535994 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 16:32:21.536304 23339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:21.525645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39811 (local address 127.19.238.65:42713)
0812 16:32:21.525806 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:21.525847 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:21.526434 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:21.526706 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:21.526709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:21.527378 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:21.527591 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:21.527596 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:21.528473 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:21.528474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:21.529354 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:21.529357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:21.529807 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:21.529808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:21.529951 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:21.530584 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:21.530596 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:21.532750 (+ 2154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:21.534606 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:21.534616 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:21.534618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:21.534888 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:21.535116 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:21.535117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:21.535117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:21.535248 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:21.535541 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:21.535550 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:21.535705 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 16:32:21.535826 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 16:32:21.536044 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260812 16:32:21.536854 23337 heartbeater.cc:347] Connected to a master server at 127.19.238.126:39811
I20260812 16:32:21.536972 23337 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:21.537199 23337 heartbeater.cc:511] Master 127.19.238.126:39811 requested a full tablet report, sending...
I20260812 16:32:21.537847 23112 ts_manager.cc:194] Registered new tserver with Master: 85eee359aed148849177ffffcd442c7f (127.19.238.65:46311)
I20260812 16:32:21.538822 23112 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:42713
WARNING: no policy specified for kudu/127.19.238.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
I20260812 16:32:21.561178 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:39811
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:34833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:21.649111 23342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:21.649338 23342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:21.649362 23342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:21.650575 23342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:21.650622 23342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:21.650678 23342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:32:21.652228 23342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:34833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:39811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:21.652514 23342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:21.652786 23342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:21.658104 23348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:21.658134 23350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:21.658141 23347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:21.658147 23342 server_base.cc:1034] running on GCE node
I20260812 16:32:21.658555 23342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:21.658789 23342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:21.659961 23342 hybrid_clock.cc:648] HybridClock initialized: now 1786552341659949 us; error 39 us; skew 500 ppm
Aug 12 16:32:21 dist-test-slave-mvvj krb5kdc[23060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552341, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:21.662706 23342 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:32:21.663271 23342 webserver.cc:460] Webserver started at http://127.19.238.66:41917/ using document root <none> and password file <none>
I20260812 16:32:21.663552 23342 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:21.663625 23342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:21.663769 23342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:21.664911 23342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "8ec7e64e665f4a90b09b5a3d2c6a9b6e"
format_stamp: "Formatted at 2026-08-12 16:32:21 on dist-test-slave-mvvj"
server_key: "d2122cceaa8795d58e497c6f2e782c44"
server_key_iv: "e0a115dc578419349f0b79023f53bcf7"
server_key_version: "encryptionkey@0"
I20260812 16:32:21.665293 23342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "8ec7e64e665f4a90b09b5a3d2c6a9b6e"
format_stamp: "Formatted at 2026-08-12 16:32:21 on dist-test-slave-mvvj"
server_key: "d2122cceaa8795d58e497c6f2e782c44"
server_key_iv: "e0a115dc578419349f0b79023f53bcf7"
server_key_version: "encryptionkey@0"
I20260812 16:32:21.666810 23342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:32:21.667699 23357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:21.667976 23342 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:21.668097 23342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "8ec7e64e665f4a90b09b5a3d2c6a9b6e"
format_stamp: "Formatted at 2026-08-12 16:32:21 on dist-test-slave-mvvj"
server_key: "d2122cceaa8795d58e497c6f2e782c44"
server_key_iv: "e0a115dc578419349f0b79023f53bcf7"
server_key_version: "encryptionkey@0"
I20260812 16:32:21.668183 23342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:21.678192 23342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:21.680590 23342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:21.680764 23342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:21.681016 23342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:21.681357 23342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:21.681411 23342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:21.681457 23342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:21.681499 23342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:21.688015 23342 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:36529
I20260812 16:32:21.688082 23482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:36529 every 8 connection(s)
I20260812 16:32:21.688444 23342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
Aug 12 16:32:21 dist-test-slave-mvvj krb5kdc[23060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552341, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:21.697278 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 23342
I20260812 16:32:21.697394 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:32:21.697665 20409 external_mini_cluster.cc:1428] Setting key f83806e480adbfffa46356450452066e
I20260812 16:32:21.699986 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:21.689415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:59503 (local address 127.19.238.126:39811)
0812 16:32:21.689508 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:21.689512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:21.689716 (+ 204us) server_negotiation.cc:407] Connection header received
0812 16:32:21.690374 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:21.690375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:21.690389 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:21.690470 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:21.691584 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:21.692206 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:21.693341 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:21.693568 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:21.693667 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:21.696431 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:21.696433 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:21.696433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:21.696441 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:21.698146 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:21.698745 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:21.698746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:21.698747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:21.698817 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:21.699152 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:21.699154 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:21.699155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:21.699288 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:21.699359 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:21.699730 (+ 371us) server_negotiation.cc:299] Negotiation successful
0812 16:32:21.699820 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 16:32:21.700229 23485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:21.689384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39811 (local address 127.19.238.66:59503)
0812 16:32:21.689580 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:21.689624 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:21.690241 (+ 617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:21.690550 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:21.690554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:21.691219 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:21.691439 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:21.691443 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:21.692315 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:21.692316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:21.693228 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:21.693231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:21.693680 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:21.693681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:21.693831 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:21.694461 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:21.694473 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:21.696313 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:21.698230 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:21.698240 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:21.698242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:21.698636 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:21.698896 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:21.698897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:21.698897 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:21.699057 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:21.699364 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:21.699372 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:21.699569 (+ 197us) client_negotiation.cc:769] Sending connection context
0812 16:32:21.699697 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 16:32:21.699930 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":92,"threads_started":1}
I20260812 16:32:21.700745 23483 heartbeater.cc:347] Connected to a master server at 127.19.238.126:39811
I20260812 16:32:21.700860 23483 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:21.701061 23483 heartbeater.cc:511] Master 127.19.238.126:39811 requested a full tablet report, sending...
I20260812 16:32:21.701530 23112 ts_manager.cc:194] Registered new tserver with Master: 8ec7e64e665f4a90b09b5a3d2c6a9b6e (127.19.238.66:36529)
I20260812 16:32:21.701998 23112 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:59503
WARNING: no policy specified for kudu/127.19.238.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
I20260812 16:32:21.727608 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:39811
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:34833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:21.813233 23489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:21.813474 23489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:21.813503 23489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:21.814842 23489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:21.814894 23489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:21.814924 23489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:32:21.816607 23489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:34833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:39811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:21.816902 23489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:21.817157 23489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:21.822594 23495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:21.822571 23498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:21.822567 23496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:21.822666 23489 server_base.cc:1034] running on GCE node
I20260812 16:32:21.823066 23489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:21.823376 23489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:21.824560 23489 hybrid_clock.cc:648] HybridClock initialized: now 1786552341824533 us; error 45 us; skew 500 ppm
Aug 12 16:32:21 dist-test-slave-mvvj krb5kdc[23060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552341, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:21.827107 23489 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:32:21.827579 23489 webserver.cc:460] Webserver started at http://127.19.238.67:40137/ using document root <none> and password file <none>
I20260812 16:32:21.827822 23489 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:21.827937 23489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:21.828095 23489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:21.829246 23489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "c645a61c64aa4e588c507f2e6a45c2ad"
format_stamp: "Formatted at 2026-08-12 16:32:21 on dist-test-slave-mvvj"
server_key: "56f8262dd5b7672a718874f1aacf85cc"
server_key_iv: "fea491b2ab1eac65cee6321b81dd385b"
server_key_version: "encryptionkey@0"
I20260812 16:32:21.829656 23489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "c645a61c64aa4e588c507f2e6a45c2ad"
format_stamp: "Formatted at 2026-08-12 16:32:21 on dist-test-slave-mvvj"
server_key: "56f8262dd5b7672a718874f1aacf85cc"
server_key_iv: "fea491b2ab1eac65cee6321b81dd385b"
server_key_version: "encryptionkey@0"
I20260812 16:32:21.831074 23489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:32:21.831911 23508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:21.832091 23489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:21.832190 23489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "c645a61c64aa4e588c507f2e6a45c2ad"
format_stamp: "Formatted at 2026-08-12 16:32:21 on dist-test-slave-mvvj"
server_key: "56f8262dd5b7672a718874f1aacf85cc"
server_key_iv: "fea491b2ab1eac65cee6321b81dd385b"
server_key_version: "encryptionkey@0"
I20260812 16:32:21.832273 23489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:21.843434 23489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:21.845728 23489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:21.845927 23489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:21.846197 23489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:21.846578 23489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:21.846634 23489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:21.846678 23489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:21.846710 23489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:21.853091 23489 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:46725
I20260812 16:32:21.853132 23633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:46725 every 8 connection(s)
I20260812 16:32:21.853497 23489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
Aug 12 16:32:21 dist-test-slave-mvvj krb5kdc[23060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552341, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:21.862905 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:21.854235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:49247 (local address 127.19.238.126:39811)
0812 16:32:21.854280 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:21.854282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:21.854527 (+ 245us) server_negotiation.cc:407] Connection header received
0812 16:32:21.855117 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:21.855118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:21.855129 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:21.855187 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:21.856193 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:21.856690 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:21.857681 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:21.857894 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:21.857962 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:21.860217 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:21.860219 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:21.860219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:21.860227 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:21.861454 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:21.861844 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:21.861845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:21.861845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:21.861894 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:21.862139 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:21.862140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:21.862140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:21.862260 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:21.862321 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:21.862750 (+ 429us) server_negotiation.cc:299] Negotiation successful
0812 16:32:21.862832 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:32:21.863075 23639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:21.854209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39811 (local address 127.19.238.67:49247)
0812 16:32:21.854373 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:21.854411 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:21.855014 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:21.855242 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:21.855245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:21.855890 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:21.856107 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:21.856112 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:21.856776 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:21.856776 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:21.857584 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:21.857586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:21.857981 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:21.857982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:21.858088 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:21.858625 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:21.858633 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:21.860112 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:21.861538 (+ 1426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:21.861545 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:21.861547 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:21.861780 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:21.861952 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:21.861953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:21.861953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:21.862052 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:21.862423 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:21.862430 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:21.862575 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 16:32:21.862693 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 16:32:21.862835 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
I20260812 16:32:21.863662 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 23489
I20260812 16:32:21.863690 23635 heartbeater.cc:347] Connected to a master server at 127.19.238.126:39811
I20260812 16:32:21.863770 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:32:21.863793 23635 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:21.864001 20409 external_mini_cluster.cc:1428] Setting key 7cd20c07ff9d4d005ba25edb80e5afe6
I20260812 16:32:21.864033 23635 heartbeater.cc:511] Master 127.19.238.126:39811 requested a full tablet report, sending...
I20260812 16:32:21.864454 23111 ts_manager.cc:194] Registered new tserver with Master: c645a61c64aa4e588c507f2e6a45c2ad (127.19.238.67:46725)
I20260812 16:32:21.864850 23111 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:49247
I20260812 16:32:21.864854 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 16:32:21.877974 23164 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:21.876804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33336 (local address 127.19.238.126:39811)
0812 16:32:21.876898 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:21.876901 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:21.877056 (+ 155us) server_negotiation.cc:407] Connection header received
0812 16:32:21.877146 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:21.877147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:21.877163 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:21.877222 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:21.877890 (+ 668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33336: BlockingRecv error: recv got EOF from 127.0.0.1:33336 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 16:32:21.878872 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 23188
I20260812 16:32:21.887362 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 23342
I20260812 16:32:21.894186 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 23489
I20260812 16:32:21.902872 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 23076
2026-08-12T16:32:21Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2022 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:32:23 dist-test-slave-mvvj krb5kdc[23650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:32:23 dist-test-slave-mvvj krb5kdc[23650](info): set up 2 sockets
Aug 12 16:32:23 dist-test-slave-mvvj krb5kdc[23650](info): commencing operation
krb5kdc: starting...
W20260812 16:32:24.347163 20409 mini_kdc.cc:121] Time spent starting KDC: real 2.421s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:32:24 dist-test-slave-mvvj krb5kdc[23650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:32:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:32:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:32:24.474655 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:36471
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:45255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:36471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:24.557906 23664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:24.558117 23664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:24.558140 23664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:24.559331 23664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:24.559378 23664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:24.559398 23664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:24.559412 23664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:24.559461 23664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:32:24.561034 23664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:45255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:36471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:36471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:24.561297 23664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:24.561540 23664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:24.566566 23669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:24.566650 23664 server_base.cc:1034] running on GCE node
W20260812 16:32:24.566628 23672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:24.566628 23670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:24.567097 23664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:24.567386 23664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:24.568553 23664 hybrid_clock.cc:648] HybridClock initialized: now 1786552344568550 us; error 51 us; skew 500 ppm
Aug 12 16:32:24 dist-test-slave-mvvj krb5kdc[23650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552344, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:24.577456 23664 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:32:24.577991 23664 webserver.cc:460] Webserver started at http://127.19.238.126:37317/ using document root <none> and password file <none>
I20260812 16:32:24.578248 23664 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:24.578325 23664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:24.578481 23664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:24.579622 23664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "db64de114b3145868734d3a24c3fc5b1"
format_stamp: "Formatted at 2026-08-12 16:32:24 on dist-test-slave-mvvj"
server_key: "17b6c87342c819389d9beb3e2456839e"
server_key_iv: "cd3c75620b20e8f582f8143d2e3a2a9a"
server_key_version: "encryptionkey@0"
I20260812 16:32:24.580017 23664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "db64de114b3145868734d3a24c3fc5b1"
format_stamp: "Formatted at 2026-08-12 16:32:24 on dist-test-slave-mvvj"
server_key: "17b6c87342c819389d9beb3e2456839e"
server_key_iv: "cd3c75620b20e8f582f8143d2e3a2a9a"
server_key_version: "encryptionkey@0"
I20260812 16:32:24.581374 23664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:24.582154 23680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:24.582333 23664 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:24.582470 23664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "db64de114b3145868734d3a24c3fc5b1"
format_stamp: "Formatted at 2026-08-12 16:32:24 on dist-test-slave-mvvj"
server_key: "17b6c87342c819389d9beb3e2456839e"
server_key_iv: "cd3c75620b20e8f582f8143d2e3a2a9a"
server_key_version: "encryptionkey@0"
I20260812 16:32:24.582552 23664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:24.601929 23664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:24.604012 23664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:24.604177 23664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:24.608577 23664 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:36471
I20260812 16:32:24.608636 23741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:36471 every 8 connection(s)
I20260812 16:32:24.609200 23664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:32:24.609928 23742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:24.609964 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 23664
I20260812 16:32:24.610081 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:32:24.610266 20409 external_mini_cluster.cc:1428] Setting key 3d9ce25968e23312b7b1c1140e7ca9b4
I20260812 16:32:24.613343 23742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1: Bootstrap starting.
I20260812 16:32:24.614107 23742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:24.614430 23742 log.cc:824] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:24.615363 23742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1: No bootstrap required, opened a new log
Aug 12 16:32:24 dist-test-slave-mvvj krb5kdc[23650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:24.617698 23742 raft_consensus.cc:348] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db64de114b3145868734d3a24c3fc5b1" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 36471 } }
I20260812 16:32:24.617865 23742 raft_consensus.cc:374] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:24.617889 23742 raft_consensus.cc:729] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db64de114b3145868734d3a24c3fc5b1, State: Initialized, Role: FOLLOWER
I20260812 16:32:24.618042 23742 consensus_queue.cc:260] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db64de114b3145868734d3a24c3fc5b1" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 36471 } }
I20260812 16:32:24.618111 23742 raft_consensus.cc:388] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:32:24.618144 23742 raft_consensus.cc:482] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:32:24.618188 23742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:24.618755 23742 raft_consensus.cc:504] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db64de114b3145868734d3a24c3fc5b1" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 36471 } }
I20260812 16:32:24.618892 23742 leader_election.cc:302] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db64de114b3145868734d3a24c3fc5b1; no voters:
I20260812 16:32:24.619096 23742 leader_election.cc:288] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:32:24.619131 23750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:24.619266 23750 raft_consensus.cc:686] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1 [term 1 LEADER]: Becoming Leader. State: Replica: db64de114b3145868734d3a24c3fc5b1, State: Running, Role: LEADER
I20260812 16:32:24.619395 23750 consensus_queue.cc:237] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db64de114b3145868734d3a24c3fc5b1" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 36471 } }
I20260812 16:32:24.619426 23742 sys_catalog.cc:564] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:24.619807 23752 sys_catalog.cc:455] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db64de114b3145868734d3a24c3fc5b1. Latest consensus state: current_term: 1 leader_uuid: "db64de114b3145868734d3a24c3fc5b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db64de114b3145868734d3a24c3fc5b1" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 36471 } } }
I20260812 16:32:24.619918 23752 sys_catalog.cc:458] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:24.620142 23754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:32:24.620767 23754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:32:24.621946 23751 sys_catalog.cc:455] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db64de114b3145868734d3a24c3fc5b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db64de114b3145868734d3a24c3fc5b1" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 36471 } } }
I20260812 16:32:24.622119 23751 sys_catalog.cc:458] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:24.622550 23746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:24.610845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53796 (local address 127.19.238.126:36471)
0812 16:32:24.611030 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:24.611038 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:24.611063 (+ 25us) server_negotiation.cc:407] Connection header received
0812 16:32:24.611530 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:24.611537 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:24.611583 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:24.611698 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:24.612659 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:24.613611 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:24.614877 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:24.615295 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:24.615427 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:24.618246 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:24.618250 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:24.618251 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:24.618281 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:24.620417 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:24.620890 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:24.620894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:24.620894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:24.621009 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:24.621232 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:24.621233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:24.621233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:24.621565 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:24.621721 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:24.622194 (+ 473us) server_negotiation.cc:299] Negotiation successful
0812 16:32:24.622341 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
I20260812 16:32:24.623081 23754 catalog_manager.cc:1347] Generated new cluster ID: 3add65a9fabb48ddbbe381ffdabdc183
I20260812 16:32:24.623138 23754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:32:24.623816 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:32:24.637017 23754 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:32:24.637770 23754 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:24.643390 23746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:24.639484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53802 (local address 127.19.238.126:36471)
0812 16:32:24.639534 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:24.639536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:24.639665 (+ 129us) server_negotiation.cc:407] Connection header received
0812 16:32:24.639748 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:24.639749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:24.639760 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:24.639857 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:24.640239 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:24.640840 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:24.641870 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:24.642092 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:24.642178 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:24.642910 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:24.642912 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:24.642913 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:32:24.643117 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:24.643136 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:32:24.643142 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:24.643202 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:24.643268 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 16:32:24.643299 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:32:24.645846 23754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db64de114b3145868734d3a24c3fc5b1: Generated new TSK 0
I20260812 16:32:24.646193 23754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:32:24.649744 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 23664
2026-08-12T16:32:24Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2744 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:32:24 dist-test-slave-mvvj krb5kdc[23782](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:32:24 dist-test-slave-mvvj krb5kdc[23782](info): set up 2 sockets
Aug 12 16:32:24 dist-test-slave-mvvj krb5kdc[23782](info): commencing operation
krb5kdc: starting...
W20260812 16:32:25.800035 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:32:25 dist-test-slave-mvvj krb5kdc[23782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:32:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:32:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:32:25.927693 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:45659
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:45659
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:26.012100 23797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:26.012372 23797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:26.012413 23797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:26.013607 23797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:26.013688 23797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:26.013713 23797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:26.013732 23797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:26.013767 23797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:32:26.015344 23797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:45659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:45659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:26.015669 23797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:26.015952 23797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:26.021286 23803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:26.021337 23806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:26.021384 23804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:26.021533 23797 server_base.cc:1034] running on GCE node
I20260812 16:32:26.021823 23797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:26.022110 23797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:26.023248 23797 hybrid_clock.cc:648] HybridClock initialized: now 1786552346023241 us; error 34 us; skew 500 ppm
Aug 12 16:32:26 dist-test-slave-mvvj krb5kdc[23782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552346, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:26.026116 23797 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:32:26.026587 23797 webserver.cc:460] Webserver started at http://127.19.238.126:33517/ using document root <none> and password file <none>
I20260812 16:32:26.026834 23797 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:26.026882 23797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:26.027014 23797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:26.028209 23797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "fda86e0f93eb4a649c58329c1fd307e7"
format_stamp: "Formatted at 2026-08-12 16:32:26 on dist-test-slave-mvvj"
server_key: "6ffa5c717c042cfa78a1902477fb21b7"
server_key_iv: "7c022b91965dbb95492ece4016dfbcfb"
server_key_version: "encryptionkey@0"
I20260812 16:32:26.028638 23797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "fda86e0f93eb4a649c58329c1fd307e7"
format_stamp: "Formatted at 2026-08-12 16:32:26 on dist-test-slave-mvvj"
server_key: "6ffa5c717c042cfa78a1902477fb21b7"
server_key_iv: "7c022b91965dbb95492ece4016dfbcfb"
server_key_version: "encryptionkey@0"
I20260812 16:32:26.030110 23797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:32:26.030938 23814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:26.031162 23797 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:26.031267 23797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "fda86e0f93eb4a649c58329c1fd307e7"
format_stamp: "Formatted at 2026-08-12 16:32:26 on dist-test-slave-mvvj"
server_key: "6ffa5c717c042cfa78a1902477fb21b7"
server_key_iv: "7c022b91965dbb95492ece4016dfbcfb"
server_key_version: "encryptionkey@0"
I20260812 16:32:26.031342 23797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:26.044955 23797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:26.047219 23797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:26.047343 23797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:26.051507 23797 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:45659
I20260812 16:32:26.051585 23879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:45659 every 8 connection(s)
I20260812 16:32:26.052017 23797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:32:26.053274 23881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:26.053543 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 23797
I20260812 16:32:26.053625 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:32:26.053828 20409 external_mini_cluster.cc:1428] Setting key 45d0765b562e06d0528bba0e5dd10b9d
I20260812 16:32:26.056871 23881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7: Bootstrap starting.
I20260812 16:32:26.057663 23881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:26.057981 23881 log.cc:824] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:26.058825 23881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7: No bootstrap required, opened a new log
Aug 12 16:32:26 dist-test-slave-mvvj krb5kdc[23782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:26.061223 23881 raft_consensus.cc:348] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda86e0f93eb4a649c58329c1fd307e7" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45659 } }
I20260812 16:32:26.061393 23881 raft_consensus.cc:374] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:26.061456 23881 raft_consensus.cc:729] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fda86e0f93eb4a649c58329c1fd307e7, State: Initialized, Role: FOLLOWER
I20260812 16:32:26.061584 23881 consensus_queue.cc:260] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda86e0f93eb4a649c58329c1fd307e7" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45659 } }
I20260812 16:32:26.061657 23881 raft_consensus.cc:388] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:32:26.061697 23881 raft_consensus.cc:482] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:32:26.061745 23881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:26.062515 23881 raft_consensus.cc:504] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda86e0f93eb4a649c58329c1fd307e7" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45659 } }
I20260812 16:32:26.062611 23881 leader_election.cc:302] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fda86e0f93eb4a649c58329c1fd307e7; no voters:
I20260812 16:32:26.062809 23881 leader_election.cc:288] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:32:26.063068 23881 sys_catalog.cc:564] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:26.064097 23886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:26.064339 23886 raft_consensus.cc:686] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7 [term 1 LEADER]: Becoming Leader. State: Replica: fda86e0f93eb4a649c58329c1fd307e7, State: Running, Role: LEADER
I20260812 16:32:26.064498 23886 consensus_queue.cc:237] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda86e0f93eb4a649c58329c1fd307e7" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45659 } }
I20260812 16:32:26.065089 23887 sys_catalog.cc:455] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fda86e0f93eb4a649c58329c1fd307e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda86e0f93eb4a649c58329c1fd307e7" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45659 } } }
I20260812 16:32:26.065177 23887 sys_catalog.cc:458] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:26.065326 23887 sys_catalog.cc:455] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fda86e0f93eb4a649c58329c1fd307e7. Latest consensus state: current_term: 1 leader_uuid: "fda86e0f93eb4a649c58329c1fd307e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda86e0f93eb4a649c58329c1fd307e7" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45659 } } }
I20260812 16:32:26.065387 23887 sys_catalog.cc:458] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:26.065557 23902 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:32:26.066018 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:26.054417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40662 (local address 127.19.238.126:45659)
0812 16:32:26.054629 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:26.054640 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:26.054668 (+ 28us) server_negotiation.cc:407] Connection header received
0812 16:32:26.055114 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:26.055121 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:26.055170 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:26.055297 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:26.056166 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:26.057137 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:26.058224 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:26.058782 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:26.058929 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:26.061838 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:26.061842 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:26.061842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:26.061855 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:26.064000 (+ 2145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:26.064501 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:26.064505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:26.064505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:26.064615 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:26.064856 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:26.064858 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:26.064858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:26.065217 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:26.065386 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:26.065618 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 16:32:26.065779 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":77,"threads_started":1}
I20260812 16:32:26.066144 23902 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:32:26.067997 23902 catalog_manager.cc:1347] Generated new cluster ID: 170dc329bac54c99b810ff0247f0eba3
I20260812 16:32:26.068084 23902 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:32:26.076155 23902 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:32:26.076743 23902 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:26.079035 23902 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fda86e0f93eb4a649c58329c1fd307e7: Generated new TSK 0
I20260812 16:32:26.079211 23902 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
I20260812 16:32:26.113692 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:45659
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:26.237615 23910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:26.237910 23910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:26.237944 23910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:26.239687 23910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:26.239751 23910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:26.239792 23910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:32:26.241981 23910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:45659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:26.242238 23910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:26.242494 23910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:26.248526 23920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:26.248718 23917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:26.248749 23910 server_base.cc:1034] running on GCE node
W20260812 16:32:26.248724 23918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:26.249162 23910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:26.249428 23910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:26.250581 23910 hybrid_clock.cc:648] HybridClock initialized: now 1786552346250559 us; error 40 us; skew 500 ppm
Aug 12 16:32:26 dist-test-slave-mvvj krb5kdc[23782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552346, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:26.253522 23910 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:32:26.253969 23910 webserver.cc:460] Webserver started at http://127.19.238.65:36085/ using document root <none> and password file <none>
I20260812 16:32:26.254199 23910 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:26.254266 23910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:26.254412 23910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:26.255584 23910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "d066bee0f389468b8381328d6287cd12"
format_stamp: "Formatted at 2026-08-12 16:32:26 on dist-test-slave-mvvj"
server_key: "91782a7735b722c21bd423e0ce2634ac"
server_key_iv: "9a2e4f0f108c7093ecf6906cf67f2787"
server_key_version: "encryptionkey@0"
I20260812 16:32:26.255998 23910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "d066bee0f389468b8381328d6287cd12"
format_stamp: "Formatted at 2026-08-12 16:32:26 on dist-test-slave-mvvj"
server_key: "91782a7735b722c21bd423e0ce2634ac"
server_key_iv: "9a2e4f0f108c7093ecf6906cf67f2787"
server_key_version: "encryptionkey@0"
I20260812 16:32:26.257467 23910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:26.258351 23928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:26.258556 23910 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:26.258677 23910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "d066bee0f389468b8381328d6287cd12"
format_stamp: "Formatted at 2026-08-12 16:32:26 on dist-test-slave-mvvj"
server_key: "91782a7735b722c21bd423e0ce2634ac"
server_key_iv: "9a2e4f0f108c7093ecf6906cf67f2787"
server_key_version: "encryptionkey@0"
I20260812 16:32:26.258759 23910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:26.277448 23910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:26.279484 23910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:26.279654 23910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:26.279984 23910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:26.280326 23910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:26.280380 23910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:26.280424 23910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:26.280448 23910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:26.286628 23910 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:36791
I20260812 16:32:26.286744 24061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:36791 every 8 connection(s)
I20260812 16:32:26.287067 23910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:32:26.290802 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 23910
I20260812 16:32:26.290916 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:32:26.291133 20409 external_mini_cluster.cc:1428] Setting key bb52005d1f9d08e831fe09cae40c1e86
Aug 12 16:32:26 dist-test-slave-mvvj krb5kdc[23782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552346, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:26.297488 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:26.287920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:36667 (local address 127.19.238.126:45659)
0812 16:32:26.287983 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:26.287985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:26.288208 (+ 223us) server_negotiation.cc:407] Connection header received
0812 16:32:26.288740 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:26.288741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:26.288750 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:26.288811 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:26.289877 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:26.290420 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:26.291905 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:26.292136 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:26.292227 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:26.294589 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:26.294590 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:26.294591 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:26.294598 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:26.296064 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:26.296497 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:26.296498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:26.296498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:26.296557 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:26.296807 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:26.296808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:26.296809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:26.296937 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:26.297006 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:26.297291 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 16:32:26.297383 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 16:32:26.297698 24064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:26.287884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:45659 (local address 127.19.238.65:36667)
0812 16:32:26.288066 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:26.288114 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:26.288651 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:26.288878 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:26.288881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:26.289550 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:26.289752 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:26.289758 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:26.290508 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:26.290509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:26.291714 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:26.291718 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:26.292243 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:26.292244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:26.292394 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:26.292971 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:26.292979 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:26.294478 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:26.296172 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:26.296180 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:26.296182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:26.296420 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:26.296631 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:26.296632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:26.296632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:26.296744 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:26.297010 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:26.297018 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:26.297159 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 16:32:26.297269 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 16:32:26.297441 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":97,"threads_started":1}
I20260812 16:32:26.298372 24062 heartbeater.cc:347] Connected to a master server at 127.19.238.126:45659
I20260812 16:32:26.298485 24062 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:26.298673 24062 heartbeater.cc:511] Master 127.19.238.126:45659 requested a full tablet report, sending...
I20260812 16:32:26.299202 23835 ts_manager.cc:194] Registered new tserver with Master: d066bee0f389468b8381328d6287cd12 (127.19.238.65:36791)
I20260812 16:32:26.299816 23835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:36667
WARNING: no policy specified for kudu/127.19.238.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
I20260812 16:32:26.319397 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:45659
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:26.403355 24068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:26.403577 24068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:26.403600 24068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:26.404785 24068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:26.404824 24068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:26.404850 24068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:32:26.406237 24068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:45659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:26.406428 24068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:26.406603 24068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:26.411615 24077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:26.411662 24074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:26.411638 24075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:26.411729 24068 server_base.cc:1034] running on GCE node
I20260812 16:32:26.412030 24068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:26.412264 24068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:26.413404 24068 hybrid_clock.cc:648] HybridClock initialized: now 1786552346413412 us; error 48 us; skew 500 ppm
Aug 12 16:32:26 dist-test-slave-mvvj krb5kdc[23782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552346, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:26.416075 24068 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:32:26.416558 24068 webserver.cc:460] Webserver started at http://127.19.238.66:45625/ using document root <none> and password file <none>
I20260812 16:32:26.416782 24068 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:26.416846 24068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:26.416992 24068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:26.418097 24068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "9de02e3cd7254e058034c19288eb1f3e"
format_stamp: "Formatted at 2026-08-12 16:32:26 on dist-test-slave-mvvj"
server_key: "0d88ff67651c91a4fe1b5f28c0b09466"
server_key_iv: "752b9178dff49bdce71b221f3e9cb70e"
server_key_version: "encryptionkey@0"
I20260812 16:32:26.418491 24068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "9de02e3cd7254e058034c19288eb1f3e"
format_stamp: "Formatted at 2026-08-12 16:32:26 on dist-test-slave-mvvj"
server_key: "0d88ff67651c91a4fe1b5f28c0b09466"
server_key_iv: "752b9178dff49bdce71b221f3e9cb70e"
server_key_version: "encryptionkey@0"
I20260812 16:32:26.419958 24068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 16:32:26.420768 24086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:26.420959 24068 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:26.421061 24068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "9de02e3cd7254e058034c19288eb1f3e"
format_stamp: "Formatted at 2026-08-12 16:32:26 on dist-test-slave-mvvj"
server_key: "0d88ff67651c91a4fe1b5f28c0b09466"
server_key_iv: "752b9178dff49bdce71b221f3e9cb70e"
server_key_version: "encryptionkey@0"
I20260812 16:32:26.421149 24068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:26.451045 24068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:26.453212 24068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:26.453387 24068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:26.453680 24068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:26.454005 24068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:26.454063 24068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:26.454109 24068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:26.454136 24068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:26.460412 24068 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:39569
I20260812 16:32:26.460485 24219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:39569 every 8 connection(s)
I20260812 16:32:26.460811 24068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:32:26.465694 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 24068
I20260812 16:32:26.465813 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:32:26.466048 20409 external_mini_cluster.cc:1428] Setting key 27a2d54d4f36bb8ed4317502ea9abe4c
Aug 12 16:32:26 dist-test-slave-mvvj krb5kdc[23782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552346, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:26.471872 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:26.461595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:48067 (local address 127.19.238.126:45659)
0812 16:32:26.461697 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:26.461700 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:26.461934 (+ 234us) server_negotiation.cc:407] Connection header received
0812 16:32:26.462475 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:26.462477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:26.462487 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:26.462556 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:26.463709 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:26.464280 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:26.465314 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:26.465503 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:26.465562 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:26.468370 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:26.468372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:26.468372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:26.468379 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:26.470128 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:26.470676 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:26.470677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:26.470678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:26.470748 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:26.471077 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:26.471078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:26.471079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:26.471236 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:26.471304 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:26.471620 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 16:32:26.471720 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 16:32:26.472116 24222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:26.461560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:45659 (local address 127.19.238.66:48067)
0812 16:32:26.461796 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:26.461839 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:26.462334 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:26.462616 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:26.462620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:26.463289 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:26.463624 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:26.463629 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:26.464359 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:26.464360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:26.465241 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:26.465243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:26.465587 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:26.465588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:26.465741 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:26.466396 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:26.466440 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:26.468253 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:26.470239 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:26.470268 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:26.470270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:26.470587 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:26.470849 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:26.470850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:26.470850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:26.470999 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:26.471312 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:26.471318 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:26.471472 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 16:32:26.471571 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 16:32:26.471755 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":124,"threads_started":1}
I20260812 16:32:26.472635 24220 heartbeater.cc:347] Connected to a master server at 127.19.238.126:45659
I20260812 16:32:26.472760 24220 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:26.472993 24220 heartbeater.cc:511] Master 127.19.238.126:45659 requested a full tablet report, sending...
I20260812 16:32:26.473562 23835 ts_manager.cc:194] Registered new tserver with Master: 9de02e3cd7254e058034c19288eb1f3e (127.19.238.66:39569)
I20260812 16:32:26.474088 23835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:48067
WARNING: no policy specified for kudu/127.19.238.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
I20260812 16:32:26.495442 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:45659
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:26.580051 24225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:26.580263 24225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:26.580287 24225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:26.581454 24225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:26.581501 24225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:26.581527 24225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:32:26.583024 24225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:45659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:26.583274 24225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:26.583511 24225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:26.588634 24233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:26.588624 24236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:26.588600 24234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:26.588896 24225 server_base.cc:1034] running on GCE node
I20260812 16:32:26.589100 24225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:26.589329 24225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:26.590514 24225 hybrid_clock.cc:648] HybridClock initialized: now 1786552346590500 us; error 38 us; skew 500 ppm
Aug 12 16:32:26 dist-test-slave-mvvj krb5kdc[23782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552346, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:26.600018 24225 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:32:26.600528 24225 webserver.cc:460] Webserver started at http://127.19.238.67:37795/ using document root <none> and password file <none>
I20260812 16:32:26.600787 24225 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:26.600854 24225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:26.601001 24225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:26.602190 24225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "5b82f3161aa64363a518c90204fd5061"
format_stamp: "Formatted at 2026-08-12 16:32:26 on dist-test-slave-mvvj"
server_key: "7af2e47533eb5bad6f449af8e0091df8"
server_key_iv: "8e5dbe1a56753767eef6bf9dd23f04a8"
server_key_version: "encryptionkey@0"
I20260812 16:32:26.602572 24225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "5b82f3161aa64363a518c90204fd5061"
format_stamp: "Formatted at 2026-08-12 16:32:26 on dist-test-slave-mvvj"
server_key: "7af2e47533eb5bad6f449af8e0091df8"
server_key_iv: "8e5dbe1a56753767eef6bf9dd23f04a8"
server_key_version: "encryptionkey@0"
I20260812 16:32:26.603956 24225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:32:26.604761 24243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:26.604928 24225 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:26.605029 24225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "5b82f3161aa64363a518c90204fd5061"
format_stamp: "Formatted at 2026-08-12 16:32:26 on dist-test-slave-mvvj"
server_key: "7af2e47533eb5bad6f449af8e0091df8"
server_key_iv: "8e5dbe1a56753767eef6bf9dd23f04a8"
server_key_version: "encryptionkey@0"
I20260812 16:32:26.605110 24225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:26.624670 24225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:26.626816 24225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:26.626976 24225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:26.627223 24225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:26.627614 24225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:26.627668 24225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:26.627712 24225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:26.627745 24225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:26.633898 24225 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:34005
I20260812 16:32:26.633949 24374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:34005 every 8 connection(s)
I20260812 16:32:26.634300 24225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
Aug 12 16:32:26 dist-test-slave-mvvj krb5kdc[23782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552346, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:26.641891 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 24225
I20260812 16:32:26.641990 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:32:26.642198 20409 external_mini_cluster.cc:1428] Setting key 50d8ce5f19c17187456eb0d2ca2337d2
I20260812 16:32:26.644214 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:26.635030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:45501 (local address 127.19.238.126:45659)
0812 16:32:26.635100 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:26.635103 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:26.635312 (+ 209us) server_negotiation.cc:407] Connection header received
0812 16:32:26.635953 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:26.635954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:26.635964 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:26.636007 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:26.636920 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:26.637422 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:26.638474 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:26.638663 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:26.638715 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:26.640992 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:26.640994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:26.640994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:26.641007 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:26.642549 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:26.643082 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:26.643084 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:26.643084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:26.643155 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:26.643485 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:26.643486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:26.643487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:26.643620 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:26.643697 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:26.644027 (+ 330us) server_negotiation.cc:299] Negotiation successful
0812 16:32:26.644119 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 16:32:26.644443 24378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:26.635029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:45659 (local address 127.19.238.67:45501)
0812 16:32:26.635204 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:26.635242 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:26.635799 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:26.636065 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:26.636068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:26.636643 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:26.636843 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:26.636847 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:26.637508 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:26.637509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:26.638394 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:26.638397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:26.638757 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:26.638759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:26.638876 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:26.639358 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:26.639414 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:26.640885 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:26.642700 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:26.642709 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:26.642711 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:26.643003 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:26.643222 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:26.643222 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:26.643223 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:26.643412 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:26.643687 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:26.643694 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:26.643873 (+ 179us) client_negotiation.cc:769] Sending connection context
0812 16:32:26.643991 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 16:32:26.644178 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260812 16:32:26.644945 24375 heartbeater.cc:347] Connected to a master server at 127.19.238.126:45659
I20260812 16:32:26.645062 24375 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:26.645249 24375 heartbeater.cc:511] Master 127.19.238.126:45659 requested a full tablet report, sending...
I20260812 16:32:26.645660 23835 ts_manager.cc:194] Registered new tserver with Master: 5b82f3161aa64363a518c90204fd5061 (127.19.238.67:34005)
I20260812 16:32:26.646121 23835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:45501
I20260812 16:32:26.654165 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 16:32:26 dist-test-slave-mvvj krb5kdc[23782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552346, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 16:32:26 dist-test-slave-mvvj krb5kdc[23782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552346, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:26.679100 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:26.671519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41126 (local address 127.19.238.126:45659)
0812 16:32:26.671596 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:26.671599 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:26.671792 (+ 193us) server_negotiation.cc:407] Connection header received
0812 16:32:26.671926 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:26.671927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:26.671945 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:26.672036 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:26.672690 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:26.673189 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:26.673962 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:26.674154 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:26.674220 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:26.676635 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:26.676636 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:26.676637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:26.676645 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:26.677979 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:26.678322 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:26.678323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:26.678323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:26.678380 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:26.678550 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:26.678550 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:26.678551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:26.678709 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:26.678787 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:26.678963 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 16:32:26.679030 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260812 16:32:26.679487 23835 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41126
I20260812 16:32:26.679989 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 23910
I20260812 16:32:26.686714 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 24068
I20260812 16:32:26.694960 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 24225
I20260812 16:32:26.701639 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 23797
2026-08-12T16:32:26Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2054 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:32:26 dist-test-slave-mvvj krb5kdc[24388](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:32:26 dist-test-slave-mvvj krb5kdc[24388](info): set up 2 sockets
Aug 12 16:32:26 dist-test-slave-mvvj krb5kdc[24388](info): commencing operation
krb5kdc: starting...
W20260812 16:32:27.843040 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:32:27 dist-test-slave-mvvj krb5kdc[24388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:32:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:32:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:32:27.976214 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:44199
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:33463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:44199
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:28.061586 24404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:28.061822 24404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:28.061846 24404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:28.063014 24404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:28.063059 24404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:28.063073 24404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:28.063086 24404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:28.063099 24404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:32:28.064697 24404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:33463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:44199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:44199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:28.064924 24404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:28.065119 24404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 16:32:28.070602 24404 server_base.cc:1034] running on GCE node
W20260812 16:32:28.070497 24415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:28.070497 24413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:28.070492 24412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:28.071067 24404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:28.071411 24404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:28.072562 24404 hybrid_clock.cc:648] HybridClock initialized: now 1786552348072537 us; error 42 us; skew 500 ppm
Aug 12 16:32:28 dist-test-slave-mvvj krb5kdc[24388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552348, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:28.075696 24404 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:32:28.076241 24404 webserver.cc:460] Webserver started at http://127.19.238.126:37923/ using document root <none> and password file <none>
I20260812 16:32:28.076481 24404 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:28.076550 24404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:28.076701 24404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:28.077797 24404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "2f2f1586a798405eab286d2a345898ed"
format_stamp: "Formatted at 2026-08-12 16:32:28 on dist-test-slave-mvvj"
server_key: "9ad90d57c1fabe7f99430ddc1b1f7da0"
server_key_iv: "07e3886b012c83219ad4d486dc523395"
server_key_version: "encryptionkey@0"
I20260812 16:32:28.078191 24404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "2f2f1586a798405eab286d2a345898ed"
format_stamp: "Formatted at 2026-08-12 16:32:28 on dist-test-slave-mvvj"
server_key: "9ad90d57c1fabe7f99430ddc1b1f7da0"
server_key_iv: "07e3886b012c83219ad4d486dc523395"
server_key_version: "encryptionkey@0"
I20260812 16:32:28.079583 24404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:32:28.080391 24423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:28.080559 24404 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:28.080657 24404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "2f2f1586a798405eab286d2a345898ed"
format_stamp: "Formatted at 2026-08-12 16:32:28 on dist-test-slave-mvvj"
server_key: "9ad90d57c1fabe7f99430ddc1b1f7da0"
server_key_iv: "07e3886b012c83219ad4d486dc523395"
server_key_version: "encryptionkey@0"
I20260812 16:32:28.080736 24404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:28.094638 24404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:28.096683 24404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:28.096858 24404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:28.101037 24404 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:44199
I20260812 16:32:28.101099 24479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:44199 every 8 connection(s)
I20260812 16:32:28.101624 24404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:32:28.101893 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 24404
I20260812 16:32:28.102010 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:32:28.102209 20409 external_mini_cluster.cc:1428] Setting key b0f3277debd09455b36927f63135578a
I20260812 16:32:28.102466 24480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:28.105976 24480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed: Bootstrap starting.
I20260812 16:32:28.106643 24480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:28.106950 24480 log.cc:824] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:28.107782 24480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed: No bootstrap required, opened a new log
Aug 12 16:32:28 dist-test-slave-mvvj krb5kdc[24388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:28.109760 24480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2f1586a798405eab286d2a345898ed" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 44199 } }
I20260812 16:32:28.109973 24480 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:28.110021 24480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f2f1586a798405eab286d2a345898ed, State: Initialized, Role: FOLLOWER
I20260812 16:32:28.110194 24480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2f1586a798405eab286d2a345898ed" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 44199 } }
I20260812 16:32:28.110286 24480 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:32:28.110327 24480 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:32:28.110376 24480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:28.110911 24480 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2f1586a798405eab286d2a345898ed" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 44199 } }
I20260812 16:32:28.111029 24480 leader_election.cc:302] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f2f1586a798405eab286d2a345898ed; no voters:
I20260812 16:32:28.111210 24480 leader_election.cc:288] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:32:28.111268 24485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:28.111420 24485 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed [term 1 LEADER]: Becoming Leader. State: Replica: 2f2f1586a798405eab286d2a345898ed, State: Running, Role: LEADER
I20260812 16:32:28.111498 24480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:28.111511 24485 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2f1586a798405eab286d2a345898ed" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 44199 } }
I20260812 16:32:28.112083 24485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f2f1586a798405eab286d2a345898ed. Latest consensus state: current_term: 1 leader_uuid: "2f2f1586a798405eab286d2a345898ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2f1586a798405eab286d2a345898ed" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 44199 } } }
I20260812 16:32:28.112176 24485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:28.112076 24486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f2f1586a798405eab286d2a345898ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2f1586a798405eab286d2a345898ed" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 44199 } } }
I20260812 16:32:28.112506 24486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:28.112949 24489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:32:28.113560 24489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:32:28.114086 24483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:28.102818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59268 (local address 127.19.238.126:44199)
0812 16:32:28.103036 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:28.103045 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:28.103081 (+ 36us) server_negotiation.cc:407] Connection header received
0812 16:32:28.103672 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:28.103682 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:28.103748 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:28.103962 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:28.104563 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:28.105958 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:28.106752 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:28.107332 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:28.107440 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:28.109822 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:28.109826 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:28.109826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:28.109838 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:28.112060 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:28.112492 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:28.112495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:28.112495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:28.112644 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:28.112921 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:28.112922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:28.112922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:28.113287 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:28.113434 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:28.113719 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 16:32:28.113879 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":102,"threads_started":1}
I20260812 16:32:28.115352 24489 catalog_manager.cc:1347] Generated new cluster ID: a75e5db7b48b4083a3d24cb9844892e2
I20260812 16:32:28.115406 24489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:32:28.126219 24489 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:32:28.126893 24489 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:28.130858 24489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f2f1586a798405eab286d2a345898ed: Generated new TSK 0
I20260812 16:32:28.131080 24489 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
I20260812 16:32:28.161054 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:44199
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:33463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:28.249421 24512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:28.249653 24512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:28.249679 24512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:28.250967 24512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:28.251014 24512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:28.251041 24512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:32:28.252694 24512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:33463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:44199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:28.252952 24512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:28.253192 24512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:28.258340 24517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:28.258339 24518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:28.258396 24521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:28.258675 24512 server_base.cc:1034] running on GCE node
I20260812 16:32:28.258898 24512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:28.259145 24512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:28.260310 24512 hybrid_clock.cc:648] HybridClock initialized: now 1786552348260298 us; error 39 us; skew 500 ppm
Aug 12 16:32:28 dist-test-slave-mvvj krb5kdc[24388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552348, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:28.263062 24512 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:32:28.263518 24512 webserver.cc:460] Webserver started at http://127.19.238.65:44997/ using document root <none> and password file <none>
I20260812 16:32:28.263767 24512 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:28.263834 24512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:28.264010 24512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:28.265405 24512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "d503f43b303145139d2a9b35abbe40ba"
format_stamp: "Formatted at 2026-08-12 16:32:28 on dist-test-slave-mvvj"
server_key: "70e05c2900f497d77139665d786b1811"
server_key_iv: "82a70b00ce9da2a24c08a27df4243e3b"
server_key_version: "encryptionkey@0"
I20260812 16:32:28.265838 24512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "d503f43b303145139d2a9b35abbe40ba"
format_stamp: "Formatted at 2026-08-12 16:32:28 on dist-test-slave-mvvj"
server_key: "70e05c2900f497d77139665d786b1811"
server_key_iv: "82a70b00ce9da2a24c08a27df4243e3b"
server_key_version: "encryptionkey@0"
I20260812 16:32:28.267500 24512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:32:28.268378 24529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:28.268631 24512 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:32:28.268739 24512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "d503f43b303145139d2a9b35abbe40ba"
format_stamp: "Formatted at 2026-08-12 16:32:28 on dist-test-slave-mvvj"
server_key: "70e05c2900f497d77139665d786b1811"
server_key_iv: "82a70b00ce9da2a24c08a27df4243e3b"
server_key_version: "encryptionkey@0"
I20260812 16:32:28.268817 24512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:28.288290 24512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:28.290701 24512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:28.290880 24512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:28.291154 24512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:28.291478 24512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:28.291534 24512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:28.291587 24512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:28.291610 24512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:28.297735 24512 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:33997
I20260812 16:32:28.297791 24666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:33997 every 8 connection(s)
I20260812 16:32:28.298146 24512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
Aug 12 16:32:28 dist-test-slave-mvvj krb5kdc[24388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552348, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:28.307513 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 24512
I20260812 16:32:28.307643 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:32:28.307901 24483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:28.298949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:57643 (local address 127.19.238.126:44199)
0812 16:32:28.299014 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:28.299017 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:28.299233 (+ 216us) server_negotiation.cc:407] Connection header received
0812 16:32:28.300272 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:28.300274 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:28.300285 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:28.300356 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:28.300991 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:28.301527 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:28.302515 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:28.302721 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:28.302782 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:28.305022 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:28.305023 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:28.305024 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:28.305031 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:28.306350 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:28.306760 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:28.306761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:28.306762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:28.306817 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:28.307097 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:28.307098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:28.307098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:28.307204 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:28.307257 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:28.307689 (+ 432us) server_negotiation.cc:299] Negotiation successful
0812 16:32:28.307784 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 16:32:28.307909 20409 external_mini_cluster.cc:1428] Setting key 5aca76032adebdfd5b134c775241323b
I20260812 16:32:28.308131 24669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:28.298921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:44199 (local address 127.19.238.65:57643)
0812 16:32:28.299093 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:28.299139 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:28.300163 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:28.300434 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:28.300437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:28.300706 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:28.300899 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:28.300902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:28.301610 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:28.301611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:28.302423 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:28.302425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:28.302802 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:28.302802 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:28.302905 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:28.303408 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:28.303416 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:28.304934 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:28.306436 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:28.306443 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:28.306445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:28.306680 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:28.306915 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:28.306916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:28.306917 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:28.307033 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:28.307322 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:28.307332 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:28.307518 (+ 186us) client_negotiation.cc:769] Sending connection context
0812 16:32:28.307656 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 16:32:28.307874 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260812 16:32:28.308811 24667 heartbeater.cc:347] Connected to a master server at 127.19.238.126:44199
I20260812 16:32:28.308918 24667 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:28.309116 24667 heartbeater.cc:511] Master 127.19.238.126:44199 requested a full tablet report, sending...
I20260812 16:32:28.309692 24441 ts_manager.cc:194] Registered new tserver with Master: d503f43b303145139d2a9b35abbe40ba (127.19.238.65:33997)
I20260812 16:32:28.310622 24441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:57643
WARNING: no policy specified for kudu/127.19.238.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
I20260812 16:32:28.339148 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:44199
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:33463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:28.422945 24672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:28.423159 24672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:28.423183 24672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:28.424321 24672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:28.424371 24672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:28.424397 24672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:32:28.425887 24672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:33463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:44199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:28.426086 24672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:28.426333 24672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:28.431658 24677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:28.431667 24672 server_base.cc:1034] running on GCE node
W20260812 16:32:28.431632 24678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:28.431632 24680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:28.432150 24672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:28.432433 24672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:28.433594 24672 hybrid_clock.cc:648] HybridClock initialized: now 1786552348433574 us; error 36 us; skew 500 ppm
Aug 12 16:32:28 dist-test-slave-mvvj krb5kdc[24388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552348, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:28.436446 24672 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:32:28.436935 24672 webserver.cc:460] Webserver started at http://127.19.238.66:45789/ using document root <none> and password file <none>
I20260812 16:32:28.437166 24672 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:28.437232 24672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:28.437376 24672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:28.438486 24672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "0662ee8a3b0648e19f2de55ec9d9a3db"
format_stamp: "Formatted at 2026-08-12 16:32:28 on dist-test-slave-mvvj"
server_key: "457fae30b8ba03717a4c7533e9d97a30"
server_key_iv: "df6faad299d753694ae285ae863cbe73"
server_key_version: "encryptionkey@0"
I20260812 16:32:28.438862 24672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "0662ee8a3b0648e19f2de55ec9d9a3db"
format_stamp: "Formatted at 2026-08-12 16:32:28 on dist-test-slave-mvvj"
server_key: "457fae30b8ba03717a4c7533e9d97a30"
server_key_iv: "df6faad299d753694ae285ae863cbe73"
server_key_version: "encryptionkey@0"
I20260812 16:32:28.440335 24672 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:32:28.441218 24691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:28.441398 24672 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:28.441511 24672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "0662ee8a3b0648e19f2de55ec9d9a3db"
format_stamp: "Formatted at 2026-08-12 16:32:28 on dist-test-slave-mvvj"
server_key: "457fae30b8ba03717a4c7533e9d97a30"
server_key_iv: "df6faad299d753694ae285ae863cbe73"
server_key_version: "encryptionkey@0"
I20260812 16:32:28.441615 24672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:28.460391 24672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:28.463104 24672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:28.463297 24672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:28.463601 24672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:28.464079 24672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:28.464144 24672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:28.464196 24672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:28.464246 24672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:28.470829 24672 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:44127
I20260812 16:32:28.470901 24819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:44127 every 8 connection(s)
I20260812 16:32:28.471298 24672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:32:28.475217 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 24672
I20260812 16:32:28.475337 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:32:28.475538 20409 external_mini_cluster.cc:1428] Setting key 6f55841a9290295b50665f19c3f3501a
Aug 12 16:32:28 dist-test-slave-mvvj krb5kdc[24388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552348, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:28.484798 24483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:28.472231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:49067 (local address 127.19.238.126:44199)
0812 16:32:28.472300 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:28.472303 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:28.472530 (+ 227us) server_negotiation.cc:407] Connection header received
0812 16:32:28.473647 (+ 1117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:28.473648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:28.473660 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:28.473740 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:28.474403 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:28.474915 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:28.478129 (+ 3214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:28.478339 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:28.478434 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:28.481262 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:28.481264 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:28.481264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:28.481273 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:28.483084 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:28.483682 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:28.483684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:28.483684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:28.483752 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:28.484101 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:28.484102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:28.484102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:28.484226 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:28.484354 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:28.484648 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 16:32:28.484725 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:32:28.485105 24822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:28.472197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:44199 (local address 127.19.238.66:49067)
0812 16:32:28.472385 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:28.472439 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:28.473505 (+ 1066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:28.473795 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:28.473798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:28.474088 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:28.474282 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:28.474286 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:28.475034 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:28.475035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:28.476354 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:28.476357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:28.478454 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:28.478455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:28.478599 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:28.479187 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:28.479195 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:28.481130 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:28.483189 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:28.483199 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:28.483203 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:28.483577 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:28.483858 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:28.483859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:28.483859 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:28.484025 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:28.484298 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:28.484305 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:28.484492 (+ 187us) client_negotiation.cc:769] Sending connection context
0812 16:32:28.484622 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 16:32:28.484811 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for kudu/127.19.238.67@KRBTEST.COM; defaulting to no policy
I20260812 16:32:28.485689 24820 heartbeater.cc:347] Connected to a master server at 127.19.238.126:44199
I20260812 16:32:28.485829 24820 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:28.486075 24820 heartbeater.cc:511] Master 127.19.238.126:44199 requested a full tablet report, sending...
I20260812 16:32:28.486524 24441 ts_manager.cc:194] Registered new tserver with Master: 0662ee8a3b0648e19f2de55ec9d9a3db (127.19.238.66:44127)
I20260812 16:32:28.487069 24441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:49067
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
I20260812 16:32:28.508594 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:44199
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:33463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:28.593840 24825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:28.594059 24825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:28.594084 24825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:28.595185 24825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:28.595225 24825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:28.595252 24825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:32:28.596779 24825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:33463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:44199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:28.596972 24825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:28.597153 24825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:28.602240 24831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:28.602232 24830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:28.602402 24833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:28.602473 24825 server_base.cc:1034] running on GCE node
I20260812 16:32:28.602726 24825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:28.602972 24825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:28.604102 24825 hybrid_clock.cc:648] HybridClock initialized: now 1786552348604084 us; error 39 us; skew 500 ppm
Aug 12 16:32:28 dist-test-slave-mvvj krb5kdc[24388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552348, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:28.606769 24825 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:32:28.607260 24825 webserver.cc:460] Webserver started at http://127.19.238.67:37613/ using document root <none> and password file <none>
I20260812 16:32:28.607488 24825 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:28.607543 24825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:28.607702 24825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:28.608855 24825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "47cb699080e24e3ab457347444c680bf"
format_stamp: "Formatted at 2026-08-12 16:32:28 on dist-test-slave-mvvj"
server_key: "a0f70787f4977c86063a46011f04809f"
server_key_iv: "25b25419e44aa74ce685627fdceb8ccd"
server_key_version: "encryptionkey@0"
I20260812 16:32:28.609218 24825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "47cb699080e24e3ab457347444c680bf"
format_stamp: "Formatted at 2026-08-12 16:32:28 on dist-test-slave-mvvj"
server_key: "a0f70787f4977c86063a46011f04809f"
server_key_iv: "25b25419e44aa74ce685627fdceb8ccd"
server_key_version: "encryptionkey@0"
I20260812 16:32:28.610625 24825 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:32:28.611563 24841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:28.611734 24825 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:28.611864 24825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "47cb699080e24e3ab457347444c680bf"
format_stamp: "Formatted at 2026-08-12 16:32:28 on dist-test-slave-mvvj"
server_key: "a0f70787f4977c86063a46011f04809f"
server_key_iv: "25b25419e44aa74ce685627fdceb8ccd"
server_key_version: "encryptionkey@0"
I20260812 16:32:28.611956 24825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:28.630040 24825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:28.632390 24825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:28.632560 24825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:28.632800 24825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:28.633149 24825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:28.633203 24825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:28.633250 24825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:28.633287 24825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:28.639372 24825 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:46287
I20260812 16:32:28.639415 24974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:46287 every 8 connection(s)
I20260812 16:32:28.639752 24825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:32:28.644979 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 24825
I20260812 16:32:28.645087 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:32:28.645301 20409 external_mini_cluster.cc:1428] Setting key 8add2daddebd56ac2c106c2b352eaab5
Aug 12 16:32:28 dist-test-slave-mvvj krb5kdc[24388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552348, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:28.649992 24483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:28.640561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:45815 (local address 127.19.238.126:44199)
0812 16:32:28.640632 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:28.640635 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:28.640846 (+ 211us) server_negotiation.cc:407] Connection header received
0812 16:32:28.641816 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:28.641817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:28.641826 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:28.641867 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:28.642469 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:28.642968 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:28.644053 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:28.644238 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:28.644288 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:28.647041 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:28.647043 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:28.647043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:28.647050 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:28.648571 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:28.649015 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:28.649016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:28.649016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:28.649071 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:28.649414 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:28.649415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:28.649415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:28.649539 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:28.649606 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:28.649861 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 16:32:28.649927 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 16:32:28.650233 24977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:28.640545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:44199 (local address 127.19.238.67:45815)
0812 16:32:28.640718 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:28.640762 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:28.641693 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:28.641932 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:28.641935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:28.642194 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:28.642395 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:28.642399 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:28.643046 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:28.643047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:28.643970 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:28.643973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:28.644334 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:28.644335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:28.644499 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:28.645037 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:28.645049 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:28.646933 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:28.648655 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:28.648661 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:28.648663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:28.648885 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:28.649155 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:28.649156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:28.649157 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:28.649277 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:28.649616 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:28.649621 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:28.649741 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 16:32:28.649829 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 16:32:28.649962 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260812 16:32:28.650771 24975 heartbeater.cc:347] Connected to a master server at 127.19.238.126:44199
I20260812 16:32:28.650898 24975 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:28.651100 24975 heartbeater.cc:511] Master 127.19.238.126:44199 requested a full tablet report, sending...
I20260812 16:32:28.651513 24441 ts_manager.cc:194] Registered new tserver with Master: 47cb699080e24e3ab457347444c680bf (127.19.238.67:46287)
I20260812 16:32:28.651916 24441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:45815
I20260812 16:32:28.657080 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 16:32:28 dist-test-slave-mvvj krb5kdc[24388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 16:32:28 dist-test-slave-mvvj krb5kdc[24388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.65@KRBTEST.COM
I20260812 16:32:28.683002 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:28.674289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46776 (local address 127.19.238.65:33997)
0812 16:32:28.674583 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:28.674593 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:28.674608 (+ 15us) server_negotiation.cc:407] Connection header received
0812 16:32:28.674638 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:28.674640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:28.674654 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:28.674729 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:28.675389 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:28.676222 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:28.677008 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:28.677346 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:28.677406 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:28.680073 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:28.680083 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:28.680084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:28.680091 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:28.681792 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:28.682174 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:28.682177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:28.682177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:28.682238 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:28.682481 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:28.682482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:28.682482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:28.682630 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:28.682739 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:28.682822 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 16:32:28.682895 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":172,"threads_started":1}
Aug 12 16:32:28 dist-test-slave-mvvj krb5kdc[24388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:28.691406 24483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:28.684581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59272 (local address 127.19.238.126:44199)
0812 16:32:28.684641 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:28.684643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:28.684775 (+ 132us) server_negotiation.cc:407] Connection header received
0812 16:32:28.684822 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:28.684824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:28.684835 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:28.684880 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:28.685358 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:28.685854 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:28.686530 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:28.686727 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:28.686783 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:28.688963 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:28.688964 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:28.688964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:28.688971 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:28.690350 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:28.690687 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:28.690688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:28.690688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:28.690739 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:28.690902 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:28.690903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:28.690903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:28.691054 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:28.691128 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:28.691253 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 16:32:28.691315 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 16:32:28.691646 24441 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59272
I20260812 16:32:28.692028 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 24512
I20260812 16:32:28.698130 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 24672
I20260812 16:32:28.707022 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 24825
I20260812 16:32:28.717475 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 24404
2026-08-12T16:32:28Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2014 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:32:28 dist-test-slave-mvvj krb5kdc[24989](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:32:28 dist-test-slave-mvvj krb5kdc[24989](info): set up 2 sockets
Aug 12 16:32:28 dist-test-slave-mvvj krb5kdc[24989](info): commencing operation
krb5kdc: starting...
W20260812 16:32:29.844820 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:32:29 dist-test-slave-mvvj krb5kdc[24989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:32:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:32:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:32:29.978955 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:40489
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:40489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:30.066128 25006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:30.066417 25006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:30.066465 25006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:30.067849 25006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:30.067925 25006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:30.067952 25006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:30.067976 25006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:30.068010 25006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:32:30.069738 25006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:40489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:40489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:30.070063 25006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:30.070335 25006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:30.076721 25011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:30.076735 25014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:30.076840 25006 server_base.cc:1034] running on GCE node
W20260812 16:32:30.076721 25012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:30.077324 25006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:30.077636 25006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:30.078801 25006 hybrid_clock.cc:648] HybridClock initialized: now 1786552350078778 us; error 35 us; skew 500 ppm
Aug 12 16:32:30 dist-test-slave-mvvj krb5kdc[24989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552350, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:30.085546 25006 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:32:30.086046 25006 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:30.086112 25006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:30.086225 25006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:30.087419 25006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "1eb2d0fc117b4588b4cc98b0ae202c26"
format_stamp: "Formatted at 2026-08-12 16:32:30 on dist-test-slave-mvvj"
server_key: "95f2dadd329dcd0de0fcf34523b45726"
server_key_iv: "71ae72fad0d5207a42feafe7f871a75a"
server_key_version: "encryptionkey@0"
I20260812 16:32:30.087780 25006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "1eb2d0fc117b4588b4cc98b0ae202c26"
format_stamp: "Formatted at 2026-08-12 16:32:30 on dist-test-slave-mvvj"
server_key: "95f2dadd329dcd0de0fcf34523b45726"
server_key_iv: "71ae72fad0d5207a42feafe7f871a75a"
server_key_version: "encryptionkey@0"
I20260812 16:32:30.089520 25006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:30.090373 25020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:30.090598 25006 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:30.090723 25006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "1eb2d0fc117b4588b4cc98b0ae202c26"
format_stamp: "Formatted at 2026-08-12 16:32:30 on dist-test-slave-mvvj"
server_key: "95f2dadd329dcd0de0fcf34523b45726"
server_key_iv: "71ae72fad0d5207a42feafe7f871a75a"
server_key_version: "encryptionkey@0"
I20260812 16:32:30.090814 25006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:30.100603 25006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:30.103344 25006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:30.103545 25006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:30.108879 25006 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:40489
I20260812 16:32:30.108954 25083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:40489 every 8 connection(s)
I20260812 16:32:30.109320 25006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:32:30.110491 25084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:30.113101 25084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26: Bootstrap starting.
I20260812 16:32:30.113761 25084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:30.114084 25084 log.cc:824] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:30.114925 25084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26: No bootstrap required, opened a new log
I20260812 16:32:30.115300 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 25006
I20260812 16:32:30.115482 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:32:30.115692 20409 external_mini_cluster.cc:1428] Setting key bfd8f0f718b7e727cad6d96f099e7d0c
I20260812 16:32:30.117448 25084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eb2d0fc117b4588b4cc98b0ae202c26" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40489 } }
I20260812 16:32:30.117645 25084 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:30.117686 25084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1eb2d0fc117b4588b4cc98b0ae202c26, State: Initialized, Role: FOLLOWER
I20260812 16:32:30.117864 25084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eb2d0fc117b4588b4cc98b0ae202c26" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40489 } }
I20260812 16:32:30.117981 25084 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:32:30.118016 25084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:32:30.118063 25084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:30.118860 25084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eb2d0fc117b4588b4cc98b0ae202c26" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40489 } }
I20260812 16:32:30.118985 25084 leader_election.cc:302] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1eb2d0fc117b4588b4cc98b0ae202c26; no voters:
I20260812 16:32:30.119155 25084 leader_election.cc:288] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:32:30.119270 25089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:30.119446 25084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:30.119470 25089 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26 [term 1 LEADER]: Becoming Leader. State: Replica: 1eb2d0fc117b4588b4cc98b0ae202c26, State: Running, Role: LEADER
I20260812 16:32:30.119629 25089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eb2d0fc117b4588b4cc98b0ae202c26" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40489 } }
I20260812 16:32:30.120007 25091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1eb2d0fc117b4588b4cc98b0ae202c26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eb2d0fc117b4588b4cc98b0ae202c26" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40489 } } }
I20260812 16:32:30.119995 25089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1eb2d0fc117b4588b4cc98b0ae202c26. Latest consensus state: current_term: 1 leader_uuid: "1eb2d0fc117b4588b4cc98b0ae202c26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eb2d0fc117b4588b4cc98b0ae202c26" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40489 } } }
I20260812 16:32:30.120113 25091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:30.120121 25089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:30.120339 25098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:32:30.120882 25098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 16:32:30 dist-test-slave-mvvj krb5kdc[24989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:30.124274 25098 catalog_manager.cc:1347] Generated new cluster ID: 8a6ce0c4ce98476d863a6cd1340be6bc
I20260812 16:32:30.124337 25098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:32:30.128638 25088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.116310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60310 (local address 127.19.238.126:40489)
0812 16:32:30.116529 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:30.116541 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:30.116579 (+ 38us) server_negotiation.cc:407] Connection header received
0812 16:32:30.117399 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:30.117417 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:30.117490 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:30.117654 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:30.118503 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.119970 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.121181 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.121726 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.121862 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:30.124666 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:30.124670 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:30.124671 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:30.124682 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:30.126699 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:30.127085 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:30.127088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:30.127089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:30.127203 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:30.127384 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:30.127385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:30.127385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:30.127700 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:30.127861 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:30.128283 (+ 422us) server_negotiation.cc:299] Negotiation successful
0812 16:32:30.128418 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":107,"threads_started":1}
I20260812 16:32:30.133992 25098 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:32:30.134760 25098 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:30.144938 25098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26: Generated new TSK 0
I20260812 16:32:30.145159 25098 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
I20260812 16:32:30.179565 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:40489
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:30.269802 25110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:30.270080 25110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:30.270129 25110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:30.271502 25110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:30.271552 25110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:30.271582 25110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:32:30.273377 25110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:40489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:30.273658 25110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:30.273940 25110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:30.279634 25120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:30.279677 25116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:30.279901 25110 server_base.cc:1034] running on GCE node
W20260812 16:32:30.279634 25117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:30.280284 25110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:30.280580 25110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:30.281772 25110 hybrid_clock.cc:648] HybridClock initialized: now 1786552350281762 us; error 45 us; skew 500 ppm
Aug 12 16:32:30 dist-test-slave-mvvj krb5kdc[24989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552350, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:30.284508 25110 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:32:30.284883 25110 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:30.284933 25110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:30.285027 25110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:30.286051 25110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "6e45bf2709c14685a165857ca17f4636"
format_stamp: "Formatted at 2026-08-12 16:32:30 on dist-test-slave-mvvj"
server_key: "f6ed1504e3b629c951f5dcfc4eb2860c"
server_key_iv: "6378fd88405ffd2111bd557bc49007cb"
server_key_version: "encryptionkey@0"
I20260812 16:32:30.286382 25110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "6e45bf2709c14685a165857ca17f4636"
format_stamp: "Formatted at 2026-08-12 16:32:30 on dist-test-slave-mvvj"
server_key: "f6ed1504e3b629c951f5dcfc4eb2860c"
server_key_iv: "6378fd88405ffd2111bd557bc49007cb"
server_key_version: "encryptionkey@0"
I20260812 16:32:30.287679 25110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 16:32:30.288494 25128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:30.288669 25110 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:32:30.288758 25110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "6e45bf2709c14685a165857ca17f4636"
format_stamp: "Formatted at 2026-08-12 16:32:30 on dist-test-slave-mvvj"
server_key: "f6ed1504e3b629c951f5dcfc4eb2860c"
server_key_iv: "6378fd88405ffd2111bd557bc49007cb"
server_key_version: "encryptionkey@0"
I20260812 16:32:30.288830 25110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:30.310272 25110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:30.312958 25110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:30.313122 25110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:30.313409 25110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:30.313752 25110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:30.313788 25110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:30.313812 25110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:30.313861 25110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:30.320088 25110 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:43683
I20260812 16:32:30.320209 25265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:43683 every 8 connection(s)
I20260812 16:32:30.320499 25110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:32:30.326189 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 25110
I20260812 16:32:30.326336 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:32:30.326529 20409 external_mini_cluster.cc:1428] Setting key dcc73f2ec99c03e37bdff6d66498ac26
Aug 12 16:32:30 dist-test-slave-mvvj krb5kdc[24989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552350, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:30.331233 25088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.321347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:57221 (local address 127.19.238.126:40489)
0812 16:32:30.321424 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:30.321426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:30.321631 (+ 205us) server_negotiation.cc:407] Connection header received
0812 16:32:30.322351 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:30.322352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:30.322368 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:30.322438 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:30.323475 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.324105 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.325103 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.325363 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.325427 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:30.327822 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:30.327823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:30.327824 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:30.327883 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:30.329478 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:30.330021 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:30.330022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:30.330023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:30.330078 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:30.330469 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:30.330470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:30.330471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:30.330634 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:30.330719 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:30.330989 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 16:32:30.331094 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 16:32:30.331256 25268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.321323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:40489 (local address 127.19.238.65:57221)
0812 16:32:30.321498 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:30.321542 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:30.322067 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:30.322526 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:30.322530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:30.322817 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:30.323362 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:30.323367 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.324202 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.324203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:30.325010 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:30.325012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.325449 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.325449 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:30.325556 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:30.326084 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:30.326092 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:30.327728 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:30.329579 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:30.329587 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:30.329590 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:30.329939 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:30.330149 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:30.330150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:30.330150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:30.330320 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:30.330715 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:30.330721 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:30.330849 (+ 128us) client_negotiation.cc:769] Sending connection context
0812 16:32:30.330942 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 16:32:30.331081 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260812 16:32:30.331921 25266 heartbeater.cc:347] Connected to a master server at 127.19.238.126:40489
I20260812 16:32:30.332026 25266 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:30.332211 25266 heartbeater.cc:511] Master 127.19.238.126:40489 requested a full tablet report, sending...
I20260812 16:32:30.332840 25043 ts_manager.cc:194] Registered new tserver with Master: 6e45bf2709c14685a165857ca17f4636 (127.19.238.65:43683)
I20260812 16:32:30.333853 25043 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:57221
WARNING: no policy specified for kudu/127.19.238.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
I20260812 16:32:30.359799 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:40489
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:30.444507 25271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:30.444767 25271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:30.444814 25271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:30.446053 25271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:30.446127 25271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:30.446172 25271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:32:30.447755 25271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:40489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:30.448076 25271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:30.448335 25271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 16:32:30.453756 25271 server_base.cc:1034] running on GCE node
W20260812 16:32:30.453719 25280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:30.453727 25278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:30.453719 25277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:30.454196 25271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:30.454504 25271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:30.455669 25271 hybrid_clock.cc:648] HybridClock initialized: now 1786552350455655 us; error 30 us; skew 500 ppm
Aug 12 16:32:30 dist-test-slave-mvvj krb5kdc[24989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552350, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:30.458380 25271 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:32:30.458789 25271 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:30.458843 25271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:30.458941 25271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:30.460067 25271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "16d96935a9c54f548f1af1823da9b2cb"
format_stamp: "Formatted at 2026-08-12 16:32:30 on dist-test-slave-mvvj"
server_key: "6a7ba2b45df7126ae46de0f6b457e2af"
server_key_iv: "c0630125086b06e4212a1f86d94394ab"
server_key_version: "encryptionkey@0"
I20260812 16:32:30.460395 25271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "16d96935a9c54f548f1af1823da9b2cb"
format_stamp: "Formatted at 2026-08-12 16:32:30 on dist-test-slave-mvvj"
server_key: "6a7ba2b45df7126ae46de0f6b457e2af"
server_key_iv: "c0630125086b06e4212a1f86d94394ab"
server_key_version: "encryptionkey@0"
I20260812 16:32:30.461671 25271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:30.462371 25287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:30.462641 25271 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:30.462747 25271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "16d96935a9c54f548f1af1823da9b2cb"
format_stamp: "Formatted at 2026-08-12 16:32:30 on dist-test-slave-mvvj"
server_key: "6a7ba2b45df7126ae46de0f6b457e2af"
server_key_iv: "c0630125086b06e4212a1f86d94394ab"
server_key_version: "encryptionkey@0"
I20260812 16:32:30.462822 25271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:30.476158 25271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:30.478345 25271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:30.478504 25271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:30.478775 25271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:30.479130 25271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:30.479168 25271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:30.479193 25271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:30.479206 25271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:30.485409 25271 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:36293
I20260812 16:32:30.485457 25422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:36293 every 8 connection(s)
I20260812 16:32:30.485735 25271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
Aug 12 16:32:30 dist-test-slave-mvvj krb5kdc[24989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552350, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:30.495566 25088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.486558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:39057 (local address 127.19.238.126:40489)
0812 16:32:30.486636 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:30.486638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:30.486852 (+ 214us) server_negotiation.cc:407] Connection header received
0812 16:32:30.487400 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:30.487402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:30.487415 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:30.487477 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:30.488501 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.489030 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.490126 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.490345 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.490436 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:30.492761 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:30.492762 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:30.492763 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:30.492771 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:30.494057 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:30.494475 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:30.494476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:30.494476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:30.494597 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:30.494866 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:30.494867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:30.494868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:30.494996 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:30.495077 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:30.495369 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 16:32:30.495463 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 16:32:30.495769 25428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.486522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:40489 (local address 127.19.238.66:39057)
0812 16:32:30.486724 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:30.486760 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:30.487285 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:30.487552 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:30.487556 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:30.487869 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:30.488391 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:30.488396 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.489140 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.489141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:30.489967 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:30.489970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.490449 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.490450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:30.490581 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:30.491150 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:30.491159 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:30.492657 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:30.494148 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:30.494158 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:30.494160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:30.494392 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:30.494680 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:30.494681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:30.494681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:30.494797 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:30.495070 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:30.495077 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:30.495231 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 16:32:30.495346 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 16:32:30.495513 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":102,"threads_started":1}
I20260812 16:32:30.495821 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 25271
I20260812 16:32:30.495955 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:32:30.496187 20409 external_mini_cluster.cc:1428] Setting key 4051889e77dd3840ce47cadc9e7dc885
I20260812 16:32:30.496591 25424 heartbeater.cc:347] Connected to a master server at 127.19.238.126:40489
I20260812 16:32:30.496688 25424 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:30.496863 25424 heartbeater.cc:511] Master 127.19.238.126:40489 requested a full tablet report, sending...
I20260812 16:32:30.497285 25043 ts_manager.cc:194] Registered new tserver with Master: 16d96935a9c54f548f1af1823da9b2cb (127.19.238.66:36293)
I20260812 16:32:30.497668 25043 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:39057
WARNING: no policy specified for kudu/127.19.238.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
I20260812 16:32:30.524111 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:40489
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:30.608371 25431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:30.608585 25431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:30.608609 25431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:30.609783 25431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:30.609829 25431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:30.609858 25431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:32:30.611419 25431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:40489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:30.611682 25431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:30.611951 25431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 16:32:30.617062 25431 server_base.cc:1034] running on GCE node
W20260812 16:32:30.617025 25437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:30.617010 25440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:30.617025 25438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:30.617483 25431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:30.617744 25431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:30.618891 25431 hybrid_clock.cc:648] HybridClock initialized: now 1786552350618870 us; error 38 us; skew 500 ppm
Aug 12 16:32:30 dist-test-slave-mvvj krb5kdc[24989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552350, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:30.621884 25431 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:32:30.622213 25431 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:30.622260 25431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:30.622357 25431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:30.623432 25431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "1d7a264495dd47dda52a94cd31e0377d"
format_stamp: "Formatted at 2026-08-12 16:32:30 on dist-test-slave-mvvj"
server_key: "c68a46bee841bd63ea67f1ffa4e3dd17"
server_key_iv: "95d459e60ce267c9092610347acf6b58"
server_key_version: "encryptionkey@0"
I20260812 16:32:30.623771 25431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "1d7a264495dd47dda52a94cd31e0377d"
format_stamp: "Formatted at 2026-08-12 16:32:30 on dist-test-slave-mvvj"
server_key: "c68a46bee841bd63ea67f1ffa4e3dd17"
server_key_iv: "95d459e60ce267c9092610347acf6b58"
server_key_version: "encryptionkey@0"
I20260812 16:32:30.625244 25431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:30.626029 25446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:30.626212 25431 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:32:30.626324 25431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "1d7a264495dd47dda52a94cd31e0377d"
format_stamp: "Formatted at 2026-08-12 16:32:30 on dist-test-slave-mvvj"
server_key: "c68a46bee841bd63ea67f1ffa4e3dd17"
server_key_iv: "95d459e60ce267c9092610347acf6b58"
server_key_version: "encryptionkey@0"
I20260812 16:32:30.626401 25431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:30.640602 25431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:30.642881 25431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:30.643059 25431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:30.643319 25431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:30.643703 25431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:30.643764 25431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:30.643811 25431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:30.643865 25431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:30.650292 25431 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:37219
I20260812 16:32:30.650329 25584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:37219 every 8 connection(s)
I20260812 16:32:30.650631 25431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
Aug 12 16:32:30 dist-test-slave-mvvj krb5kdc[24989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552350, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:30.660055 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 25431
I20260812 16:32:30.660140 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:32:30.660337 20409 external_mini_cluster.cc:1428] Setting key eca06c94c26b9749c04ddbd58ec9f73d
I20260812 16:32:30.660472 25088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.651423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:55575 (local address 127.19.238.126:40489)
0812 16:32:30.651534 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:30.651537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:30.651742 (+ 205us) server_negotiation.cc:407] Connection header received
0812 16:32:30.652358 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:30.652359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:30.652369 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:30.652421 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:30.653438 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.653948 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.655014 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.655198 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.655253 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:30.657679 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:30.657681 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:30.657681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:30.657688 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:30.659032 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:30.659418 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:30.659419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:30.659419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:30.659472 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:30.659695 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:30.659696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:30.659696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:30.659803 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:30.659914 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:30.660280 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 16:32:30.660366 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 16:32:30.660686 25587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.651406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:40489 (local address 127.19.238.67:55575)
0812 16:32:30.651607 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:30.651649 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:30.652259 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:30.652491 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:30.652495 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:30.652789 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:30.653345 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:30.653350 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.654038 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.654039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:30.654923 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:30.654926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.655274 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.655275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:30.655383 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:30.656054 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:30.656063 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:30.657582 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:30.659112 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:30.659118 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:30.659120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:30.659349 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:30.659531 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:30.659531 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:30.659532 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:30.659635 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:30.659959 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:30.659965 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:30.660093 (+ 128us) client_negotiation.cc:769] Sending connection context
0812 16:32:30.660254 (+ 161us) client_negotiation.cc:240] Negotiation successful
0812 16:32:30.660429 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":107,"threads_started":1}
I20260812 16:32:30.661252 25585 heartbeater.cc:347] Connected to a master server at 127.19.238.126:40489
I20260812 16:32:30.661360 25585 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:30.661523 25585 heartbeater.cc:511] Master 127.19.238.126:40489 requested a full tablet report, sending...
I20260812 16:32:30.661916 25043 ts_manager.cc:194] Registered new tserver with Master: 1d7a264495dd47dda52a94cd31e0377d (127.19.238.67:37219)
I20260812 16:32:30.662276 25043 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:55575
I20260812 16:32:30.672508 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:32:30.680263 25088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.673865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60316 (local address 127.19.238.126:40489)
0812 16:32:30.673947 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:30.673950 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:30.674013 (+ 63us) server_negotiation.cc:407] Connection header received
0812 16:32:30.674071 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:30.674072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:30.674082 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:30.674130 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:30.674666 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.675163 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.676201 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.676419 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.676506 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:30.677693 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:30.677694 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:30.677695 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:30.677702 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:30.679057 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:30.679415 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:30.679416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:30.679416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:30.679471 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:30.679644 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:30.679645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:30.679645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:30.679797 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:30.679970 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:30.680102 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 16:32:30.680170 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 16:32:30.682114 25043 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60316:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:32:30.683701 25043 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:32:30.690970 25597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.686455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:36293 (local address 127.0.0.1:48906)
0812 16:32:30.686590 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:30.686599 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:30.686641 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:30.687217 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:30.687217 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:30.687219 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:30.687263 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:30.687264 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.688664 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.688665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:30.689870 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:30.689873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.690582 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.690583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:30.690684 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:30.690698 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 16:32:30.690794 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 16:32:30.690853 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":26,"thread_start_us":83,"threads_started":1}
I20260812 16:32:30.691082 25598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.686254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:37219 (local address 127.0.0.1:48624)
0812 16:32:30.686421 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:30.686465 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:30.686529 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:30.687510 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:30.687511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:30.687512 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:30.687553 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:30.687554 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.688468 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.688469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:30.689870 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:30.689873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.690791 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.690792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:30.690887 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:30.690890 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:32:30.690947 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 16:32:30.690983 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":38,"thread_start_us":50,"threads_started":1}
I20260812 16:32:30.691380 25599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.686353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48624 (local address 127.19.238.67:37219)
0812 16:32:30.687308 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:30.687312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:30.687322 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:30.687356 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:30.687358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:30.687380 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:30.687512 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:30.687603 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.688354 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.689963 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.690709 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.690781 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:30.690835 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:30.690972 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 16:32:30.691014 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":912,"thread_start_us":113,"threads_started":1}
I20260812 16:32:30.691525 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.686835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48906 (local address 127.19.238.66:36293)
0812 16:32:30.687052 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:30.687057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:30.687067 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:30.687104 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:30.687106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:30.687125 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:30.687258 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:30.687338 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.688519 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.689976 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.690470 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.690580 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:30.690645 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:30.691319 (+ 674us) server_negotiation.cc:299] Negotiation successful
0812 16:32:30.691364 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":79,"threads_started":1}
I20260812 16:32:30.692034 25600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.686254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:43683 (local address 127.0.0.1:40462)
0812 16:32:30.686587 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:30.686600 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:30.686658 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:30.687014 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:30.687017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:30.687020 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:30.687105 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:30.687107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.689551 (+ 2444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.689552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:30.690493 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:30.690496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.691773 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.691774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:30.691864 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:30.691867 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:32:30.691922 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 16:32:30.691963 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":2,"thread_start_us":97,"threads_started":1}
I20260812 16:32:30.692066 25601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.686630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40462 (local address 127.19.238.65:43683)
0812 16:32:30.686815 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:30.686819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:30.686827 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:32:30.686864 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:30.686866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:30.686879 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:30.686975 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:30.687189 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.689437 (+ 2248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.690616 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.691616 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.691685 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:30.691738 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:30.691939 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 16:32:30.691979 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":99,"threads_started":1}
I20260812 16:32:30.692167 25509 tablet_service.cc:1467] Processing CreateTablet for tablet 9575c800f9a74c6f96a99cd5e0adb931 (DEFAULT_TABLE table=test-table [id=533fad2379fc4a89bf5616b7f1867eac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:30.692169 25343 tablet_service.cc:1467] Processing CreateTablet for tablet 9575c800f9a74c6f96a99cd5e0adb931 (DEFAULT_TABLE table=test-table [id=533fad2379fc4a89bf5616b7f1867eac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:30.692564 25509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9575c800f9a74c6f96a99cd5e0adb931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:30.692564 25343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9575c800f9a74c6f96a99cd5e0adb931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:30.696125 25603 tablet_bootstrap.cc:492] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb: Bootstrap starting.
I20260812 16:32:30.696799 25603 tablet_bootstrap.cc:654] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:30.696890 25188 tablet_service.cc:1467] Processing CreateTablet for tablet 9575c800f9a74c6f96a99cd5e0adb931 (DEFAULT_TABLE table=test-table [id=533fad2379fc4a89bf5616b7f1867eac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:30.697248 25188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9575c800f9a74c6f96a99cd5e0adb931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:30.697391 25603 log.cc:824] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:30.697853 25604 tablet_bootstrap.cc:492] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d: Bootstrap starting.
I20260812 16:32:30.698233 25603 tablet_bootstrap.cc:492] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb: No bootstrap required, opened a new log
I20260812 16:32:30.698290 25603 ts_tablet_manager.cc:1397] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:32:30.699748 25603 raft_consensus.cc:348] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d7a264495dd47dda52a94cd31e0377d" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37219 } } peers { permanent_uuid: "16d96935a9c54f548f1af1823da9b2cb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36293 } } peers { permanent_uuid: "6e45bf2709c14685a165857ca17f4636" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43683 } }
I20260812 16:32:30.699950 25603 raft_consensus.cc:374] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:30.699994 25603 raft_consensus.cc:729] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16d96935a9c54f548f1af1823da9b2cb, State: Initialized, Role: FOLLOWER
I20260812 16:32:30.700146 25603 consensus_queue.cc:260] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d7a264495dd47dda52a94cd31e0377d" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37219 } } peers { permanent_uuid: "16d96935a9c54f548f1af1823da9b2cb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36293 } } peers { permanent_uuid: "6e45bf2709c14685a165857ca17f4636" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43683 } }
I20260812 16:32:30.700254 25607 tablet_bootstrap.cc:492] T 9575c800f9a74c6f96a99cd5e0adb931 P 6e45bf2709c14685a165857ca17f4636: Bootstrap starting.
I20260812 16:32:30.700457 25603 ts_tablet_manager.cc:1428] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:32:30.700495 25424 heartbeater.cc:503] Master 127.19.238.126:40489 was elected leader, sending a full tablet report...
I20260812 16:32:30.701117 25604 tablet_bootstrap.cc:654] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:30.701439 25607 tablet_bootstrap.cc:654] T 9575c800f9a74c6f96a99cd5e0adb931 P 6e45bf2709c14685a165857ca17f4636: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:30.701802 25604 log.cc:824] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:30.702128 25607 log.cc:824] T 9575c800f9a74c6f96a99cd5e0adb931 P 6e45bf2709c14685a165857ca17f4636: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:30.702659 25604 tablet_bootstrap.cc:492] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d: No bootstrap required, opened a new log
I20260812 16:32:30.702739 25604 ts_tablet_manager.cc:1397] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 16:32:30.702896 25607 tablet_bootstrap.cc:492] T 9575c800f9a74c6f96a99cd5e0adb931 P 6e45bf2709c14685a165857ca17f4636: No bootstrap required, opened a new log
I20260812 16:32:30.702961 25607 ts_tablet_manager.cc:1397] T 9575c800f9a74c6f96a99cd5e0adb931 P 6e45bf2709c14685a165857ca17f4636: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:32:30.704893 25604 raft_consensus.cc:348] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d7a264495dd47dda52a94cd31e0377d" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37219 } } peers { permanent_uuid: "16d96935a9c54f548f1af1823da9b2cb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36293 } } peers { permanent_uuid: "6e45bf2709c14685a165857ca17f4636" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43683 } }
I20260812 16:32:30.704962 25607 raft_consensus.cc:348] T 9575c800f9a74c6f96a99cd5e0adb931 P 6e45bf2709c14685a165857ca17f4636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d7a264495dd47dda52a94cd31e0377d" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37219 } } peers { permanent_uuid: "16d96935a9c54f548f1af1823da9b2cb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36293 } } peers { permanent_uuid: "6e45bf2709c14685a165857ca17f4636" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43683 } }
I20260812 16:32:30.705093 25607 raft_consensus.cc:374] T 9575c800f9a74c6f96a99cd5e0adb931 P 6e45bf2709c14685a165857ca17f4636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:30.705093 25604 raft_consensus.cc:374] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:30.705130 25607 raft_consensus.cc:729] T 9575c800f9a74c6f96a99cd5e0adb931 P 6e45bf2709c14685a165857ca17f4636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e45bf2709c14685a165857ca17f4636, State: Initialized, Role: FOLLOWER
I20260812 16:32:30.705130 25604 raft_consensus.cc:729] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d7a264495dd47dda52a94cd31e0377d, State: Initialized, Role: FOLLOWER
I20260812 16:32:30.705259 25607 consensus_queue.cc:260] T 9575c800f9a74c6f96a99cd5e0adb931 P 6e45bf2709c14685a165857ca17f4636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d7a264495dd47dda52a94cd31e0377d" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37219 } } peers { permanent_uuid: "16d96935a9c54f548f1af1823da9b2cb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36293 } } peers { permanent_uuid: "6e45bf2709c14685a165857ca17f4636" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43683 } }
I20260812 16:32:30.705259 25604 consensus_queue.cc:260] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d7a264495dd47dda52a94cd31e0377d" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37219 } } peers { permanent_uuid: "16d96935a9c54f548f1af1823da9b2cb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36293 } } peers { permanent_uuid: "6e45bf2709c14685a165857ca17f4636" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43683 } }
I20260812 16:32:30.705565 25607 ts_tablet_manager.cc:1428] T 9575c800f9a74c6f96a99cd5e0adb931 P 6e45bf2709c14685a165857ca17f4636: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:32:30.705565 25604 ts_tablet_manager.cc:1428] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:32:30.705621 25266 heartbeater.cc:503] Master 127.19.238.126:40489 was elected leader, sending a full tablet report...
I20260812 16:32:30.705878 25585 heartbeater.cc:503] Master 127.19.238.126:40489 was elected leader, sending a full tablet report...
W20260812 16:32:30.736438 25427 tablet.cc:2369] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:32:30.796330 25608 raft_consensus.cc:482] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:32:30.796474 25608 raft_consensus.cc:504] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d7a264495dd47dda52a94cd31e0377d" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37219 } } peers { permanent_uuid: "16d96935a9c54f548f1af1823da9b2cb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36293 } } peers { permanent_uuid: "6e45bf2709c14685a165857ca17f4636" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43683 } }
I20260812 16:32:30.797052 25608 leader_election.cc:288] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d7a264495dd47dda52a94cd31e0377d (127.19.238.67:37219), 6e45bf2709c14685a165857ca17f4636 (127.19.238.65:43683)
I20260812 16:32:30.800058 25599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.797406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:42889 (local address 127.19.238.67:37219)
0812 16:32:30.797455 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:30.797457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:30.797466 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:32:30.797497 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:30.797498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:30.797509 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:30.797601 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:30.797782 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.798268 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.799238 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.799625 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.799715 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:30.799724 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:30.799937 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 16:32:30.799980 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:32:30.800096 25428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.797268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:37219 (local address 127.19.238.66:42889)
0812 16:32:30.797355 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:30.797368 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:30.797423 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:30.797606 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:30.797607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:30.797609 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:30.797704 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:30.797706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.798366 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.798368 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:30.799134 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:30.799136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.799712 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.799713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:30.799800 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:30.799803 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:32:30.799926 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 16:32:30.799969 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 16:32:30.800112 25601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.797343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:53875 (local address 127.19.238.65:43683)
0812 16:32:30.797409 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:30.797412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:30.797584 (+ 172us) server_negotiation.cc:407] Connection header received
0812 16:32:30.797664 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:30.797665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:30.797674 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:30.797748 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:30.797906 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.798388 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.799324 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.799697 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.799802 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:30.799811 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:30.800001 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 16:32:30.800042 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 16:32:30.800199 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.797282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:43683 (local address 127.19.238.66:53875)
0812 16:32:30.797492 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:30.797502 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:30.797600 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:30.797750 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:30.797751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:30.797753 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:30.797829 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:30.797830 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.798484 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.798486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:30.799224 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:30.799226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.799804 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.799805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:30.799901 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:30.799904 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:32:30.799982 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 16:32:30.800022 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":132,"threads_started":1}
I20260812 16:32:30.800503 25530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9575c800f9a74c6f96a99cd5e0adb931" candidate_uuid: "16d96935a9c54f548f1af1823da9b2cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d7a264495dd47dda52a94cd31e0377d" is_pre_election: true
I20260812 16:32:30.800518 25211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9575c800f9a74c6f96a99cd5e0adb931" candidate_uuid: "16d96935a9c54f548f1af1823da9b2cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e45bf2709c14685a165857ca17f4636" is_pre_election: true
I20260812 16:32:30.800688 25530 raft_consensus.cc:2454] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16d96935a9c54f548f1af1823da9b2cb in term 0.
I20260812 16:32:30.800681 25211 raft_consensus.cc:2454] T 9575c800f9a74c6f96a99cd5e0adb931 P 6e45bf2709c14685a165857ca17f4636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16d96935a9c54f548f1af1823da9b2cb in term 0.
I20260812 16:32:30.800966 25289 leader_election.cc:302] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16d96935a9c54f548f1af1823da9b2cb, 1d7a264495dd47dda52a94cd31e0377d; no voters:
I20260812 16:32:30.801127 25608 raft_consensus.cc:2781] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:32:30.801198 25608 raft_consensus.cc:482] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:32:30.801214 25608 raft_consensus.cc:3037] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:30.801941 25608 raft_consensus.cc:504] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d7a264495dd47dda52a94cd31e0377d" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37219 } } peers { permanent_uuid: "16d96935a9c54f548f1af1823da9b2cb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36293 } } peers { permanent_uuid: "6e45bf2709c14685a165857ca17f4636" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43683 } }
I20260812 16:32:30.802094 25608 leader_election.cc:288] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [CANDIDATE]: Term 1 election: Requested vote from peers 1d7a264495dd47dda52a94cd31e0377d (127.19.238.67:37219), 6e45bf2709c14685a165857ca17f4636 (127.19.238.65:43683)
I20260812 16:32:30.802234 25211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9575c800f9a74c6f96a99cd5e0adb931" candidate_uuid: "16d96935a9c54f548f1af1823da9b2cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e45bf2709c14685a165857ca17f4636"
I20260812 16:32:30.802232 25530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9575c800f9a74c6f96a99cd5e0adb931" candidate_uuid: "16d96935a9c54f548f1af1823da9b2cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d7a264495dd47dda52a94cd31e0377d"
I20260812 16:32:30.802330 25211 raft_consensus.cc:3037] T 9575c800f9a74c6f96a99cd5e0adb931 P 6e45bf2709c14685a165857ca17f4636 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:30.802341 25530 raft_consensus.cc:3037] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:30.803117 25530 raft_consensus.cc:2454] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16d96935a9c54f548f1af1823da9b2cb in term 1.
I20260812 16:32:30.803143 25211 raft_consensus.cc:2454] T 9575c800f9a74c6f96a99cd5e0adb931 P 6e45bf2709c14685a165857ca17f4636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16d96935a9c54f548f1af1823da9b2cb in term 1.
I20260812 16:32:30.803314 25291 leader_election.cc:302] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16d96935a9c54f548f1af1823da9b2cb, 6e45bf2709c14685a165857ca17f4636; no voters:
I20260812 16:32:30.803526 25608 raft_consensus.cc:2781] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:30.803691 25608 raft_consensus.cc:686] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [term 1 LEADER]: Becoming Leader. State: Replica: 16d96935a9c54f548f1af1823da9b2cb, State: Running, Role: LEADER
I20260812 16:32:30.803788 25608 consensus_queue.cc:237] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d7a264495dd47dda52a94cd31e0377d" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37219 } } peers { permanent_uuid: "16d96935a9c54f548f1af1823da9b2cb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36293 } } peers { permanent_uuid: "6e45bf2709c14685a165857ca17f4636" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43683 } }
I20260812 16:32:30.804669 25043 catalog_manager.cc:5495] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 16d96935a9c54f548f1af1823da9b2cb (127.19.238.66). New cstate: current_term: 1 leader_uuid: "16d96935a9c54f548f1af1823da9b2cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d7a264495dd47dda52a94cd31e0377d" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16d96935a9c54f548f1af1823da9b2cb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e45bf2709c14685a165857ca17f4636" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43683 } health_report { overall_health: UNKNOWN } } }
W20260812 16:32:30.821316 25267 tablet.cc:2369] T 9575c800f9a74c6f96a99cd5e0adb931 P 6e45bf2709c14685a165857ca17f4636: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:32:30.836241 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.833627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48914 (local address 127.19.238.66:36293)
0812 16:32:30.833701 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:30.833703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:30.833712 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:32:30.833758 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:30.833759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:30.833789 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:30.833869 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:30.834022 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.834518 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.835300 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:30.835519 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:30.835598 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:30.835775 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:30.835962 (+ 187us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:30.836083 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:30.836103 (+ 20us) server_negotiation.cc:299] Negotiation successful
0812 16:32:30.836125 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 16:32:30.838251 25211 raft_consensus.cc:1261] T 9575c800f9a74c6f96a99cd5e0adb931 P 6e45bf2709c14685a165857ca17f4636 [term 1 FOLLOWER]: Refusing update from remote peer 16d96935a9c54f548f1af1823da9b2cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:30.838251 25530 raft_consensus.cc:1261] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d [term 1 FOLLOWER]: Refusing update from remote peer 16d96935a9c54f548f1af1823da9b2cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:30.838685 25615 consensus_queue.cc:1035] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e45bf2709c14685a165857ca17f4636" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:30.838820 25608 consensus_queue.cc:1035] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d7a264495dd47dda52a94cd31e0377d" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:30.841976 25619 mvcc.cc:205] Tried to move back new op lower bound from 7317718429030146048 to 7317718428892991488. Current Snapshot: MvccSnapshot[applied={T|T < 7317718429030146048 or (T in {7317718429030146048})}]
I20260812 16:32:30.842125 25620 mvcc.cc:205] Tried to move back new op lower bound from 7317718429030146048 to 7317718428892991488. Current Snapshot: MvccSnapshot[applied={T|T < 7317718429030146048 or (T in {7317718429030146048})}]
I20260812 16:32:30.844223 25043 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:30.844324 25043 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:30.845155 25043 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1eb2d0fc117b4588b4cc98b0ae202c26: Sending DeleteTablet for 3 replicas of tablet 9575c800f9a74c6f96a99cd5e0adb931
I20260812 16:32:30.845507 25509 tablet_service.cc:1514] Processing DeleteTablet for tablet 9575c800f9a74c6f96a99cd5e0adb931 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:30 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:48624
I20260812 16:32:30.845616 25188 tablet_service.cc:1514] Processing DeleteTablet for tablet 9575c800f9a74c6f96a99cd5e0adb931 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:30 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:40462
I20260812 16:32:30.845767 25343 tablet_service.cc:1514] Processing DeleteTablet for tablet 9575c800f9a74c6f96a99cd5e0adb931 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:30 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:48906
I20260812 16:32:30.845803 25631 tablet_replica.cc:330] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d: stopping tablet replica
I20260812 16:32:30.845904 25631 raft_consensus.cc:2229] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:30.845988 25632 tablet_replica.cc:330] T 9575c800f9a74c6f96a99cd5e0adb931 P 6e45bf2709c14685a165857ca17f4636: stopping tablet replica
I20260812 16:32:30.846024 25631 raft_consensus.cc:2258] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:30.846099 25632 raft_consensus.cc:2229] T 9575c800f9a74c6f96a99cd5e0adb931 P 6e45bf2709c14685a165857ca17f4636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:30.846207 25632 raft_consensus.cc:2258] T 9575c800f9a74c6f96a99cd5e0adb931 P 6e45bf2709c14685a165857ca17f4636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:30.846503 25631 ts_tablet_manager.cc:1905] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:30.846513 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 25110
I20260812 16:32:30.846673 25632 ts_tablet_manager.cc:1905] T 9575c800f9a74c6f96a99cd5e0adb931 P 6e45bf2709c14685a165857ca17f4636: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:30.847108 25633 tablet_replica.cc:330] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb: stopping tablet replica
I20260812 16:32:30.848158 25633 raft_consensus.cc:2229] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:32:30.848325 25631 ts_tablet_manager.cc:1918] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:32:30.848344 25633 raft_consensus.cc:2258] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:30.848397 25631 log.cc:1196] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/9575c800f9a74c6f96a99cd5e0adb931
I20260812 16:32:30.848693 25631 ts_tablet_manager.cc:1939] T 9575c800f9a74c6f96a99cd5e0adb931 P 1d7a264495dd47dda52a94cd31e0377d: Deleting consensus metadata
I20260812 16:32:30.848867 25633 ts_tablet_manager.cc:1905] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:30.849310 25022 catalog_manager.cc:4841] TS 1d7a264495dd47dda52a94cd31e0377d (127.19.238.67:37219): tablet 9575c800f9a74c6f96a99cd5e0adb931 (table test-table [id=533fad2379fc4a89bf5616b7f1867eac]) successfully deleted
I20260812 16:32:30.851089 25633 ts_tablet_manager.cc:1918] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:32:30.851176 25633 log.cc:1196] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/9575c800f9a74c6f96a99cd5e0adb931
I20260812 16:32:30.851498 25633 ts_tablet_manager.cc:1939] T 9575c800f9a74c6f96a99cd5e0adb931 P 16d96935a9c54f548f1af1823da9b2cb: Deleting consensus metadata
I20260812 16:32:30.852097 25022 catalog_manager.cc:4841] TS 16d96935a9c54f548f1af1823da9b2cb (127.19.238.66:36293): tablet 9575c800f9a74c6f96a99cd5e0adb931 (table test-table [id=533fad2379fc4a89bf5616b7f1867eac]) successfully deleted
W20260812 16:32:30.858106 25025 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.238.65:43683 (error 108)
I20260812 16:32:30.858494 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 25271
I20260812 16:32:30.860340 25600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:30.858639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:43683 (local address 127.0.0.1:40468)
0812 16:32:30.860238 (+ 1599us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:30.860291 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1571}
W20260812 16:32:30.860657 25025 catalog_manager.cc:4584] TS 6e45bf2709c14685a165857ca17f4636 (127.19.238.65:43683): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9575c800f9a74c6f96a99cd5e0adb931: Network error: Client connection negotiation failed: client connection to 127.19.238.65:43683: connect: Connection refused (error 111)
I20260812 16:32:30.866791 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 25431
I20260812 16:32:30.873597 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 25006
2026-08-12T16:32:30Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2156 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T16:32:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:32:30Z Disabled control of system clock
I20260812 16:32:30.906535 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:40815
--webserver_interface=127.19.238.126
--webserver_port=0
--builtin_ntp_servers=127.19.238.84:41949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:40815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 16:32:30.992331 25639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:30.992556 25639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:30.992579 25639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:30.993708 25639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:30.993746 25639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:30.993759 25639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:30.993772 25639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:30.993784 25639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:32:30.995097 25639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:40815
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:40815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:30.995309 25639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:30.995565 25639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:31.001308 25645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:31.001461 25639 server_base.cc:1034] running on GCE node
W20260812 16:32:31.001288 25646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:31.001261 25648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:31.001864 25639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:31.002208 25639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:31.003367 25639 hybrid_clock.cc:648] HybridClock initialized: now 1786552351003348 us; error 38 us; skew 500 ppm
I20260812 16:32:31.005039 25639 webserver.cc:460] Webserver started at http://127.19.238.126:44467/ using document root <none> and password file <none>
I20260812 16:32:31.005318 25639 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:31.005371 25639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:31.005565 25639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:31.006794 25639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "26bad0cdb6a04767ae5d48a4610aafcd"
format_stamp: "Formatted at 2026-08-12 16:32:31 on dist-test-slave-mvvj"
server_key: "d250e20e8817a75d19fbd1836885c143"
server_key_iv: "297b4a1d315668987427e8ed3693670c"
server_key_version: "encryptionkey@0"
I20260812 16:32:31.007246 25639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "26bad0cdb6a04767ae5d48a4610aafcd"
format_stamp: "Formatted at 2026-08-12 16:32:31 on dist-test-slave-mvvj"
server_key: "d250e20e8817a75d19fbd1836885c143"
server_key_iv: "297b4a1d315668987427e8ed3693670c"
server_key_version: "encryptionkey@0"
I20260812 16:32:31.008885 25639 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260812 16:32:31.009773 25654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:31.009972 25639 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:31.010082 25639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "26bad0cdb6a04767ae5d48a4610aafcd"
format_stamp: "Formatted at 2026-08-12 16:32:31 on dist-test-slave-mvvj"
server_key: "d250e20e8817a75d19fbd1836885c143"
server_key_iv: "297b4a1d315668987427e8ed3693670c"
server_key_version: "encryptionkey@0"
I20260812 16:32:31.010169 25639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:31.014380 25639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:31.014711 25639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:31.014870 25639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:31.019527 25639 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:40815
I20260812 16:32:31.019605 25716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:40815 every 8 connection(s)
I20260812 16:32:31.020061 25639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:32:31.020803 25717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:31.021695 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 25639
I20260812 16:32:31.021782 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:32:31.021996 20409 external_mini_cluster.cc:1428] Setting key f87ac824a23d8d7733d1fba942afeb69
I20260812 16:32:31.024755 25717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd: Bootstrap starting.
I20260812 16:32:31.025077 25720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.022671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42980 (local address 127.19.238.126:40815)
0812 16:32:31.022881 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:31.022890 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:31.022930 (+ 40us) server_negotiation.cc:407] Connection header received
0812 16:32:31.023503 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:31.023512 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:31.023568 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:31.023726 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:31.024486 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:31.024490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:31.024490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:32:31.024495 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:31.024548 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:32:31.024562 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:31.024642 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:31.024775 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 16:32:31.024895 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":69,"threads_started":1}
I20260812 16:32:31.025549 25717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:31.025894 25717 log.cc:824] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:31.026702 25717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd: No bootstrap required, opened a new log
I20260812 16:32:31.028781 25717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26bad0cdb6a04767ae5d48a4610aafcd" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40815 } }
I20260812 16:32:31.028965 25717 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:31.029021 25717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26bad0cdb6a04767ae5d48a4610aafcd, State: Initialized, Role: FOLLOWER
I20260812 16:32:31.029160 25717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26bad0cdb6a04767ae5d48a4610aafcd" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40815 } }
I20260812 16:32:31.029280 25717 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:32:31.029331 25717 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:32:31.029385 25717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:31.030637 25717 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26bad0cdb6a04767ae5d48a4610aafcd" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40815 } }
I20260812 16:32:31.030792 25717 leader_election.cc:302] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26bad0cdb6a04767ae5d48a4610aafcd; no voters:
I20260812 16:32:31.031024 25717 leader_election.cc:288] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:32:31.031123 25723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:31.031286 25723 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd [term 1 LEADER]: Becoming Leader. State: Replica: 26bad0cdb6a04767ae5d48a4610aafcd, State: Running, Role: LEADER
I20260812 16:32:31.031395 25717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:31.031418 25723 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26bad0cdb6a04767ae5d48a4610aafcd" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40815 } }
I20260812 16:32:31.031941 25724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26bad0cdb6a04767ae5d48a4610aafcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26bad0cdb6a04767ae5d48a4610aafcd" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40815 } } }
I20260812 16:32:31.032059 25724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:31.032246 25725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26bad0cdb6a04767ae5d48a4610aafcd. Latest consensus state: current_term: 1 leader_uuid: "26bad0cdb6a04767ae5d48a4610aafcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26bad0cdb6a04767ae5d48a4610aafcd" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40815 } } }
I20260812 16:32:31.032603 25725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:31.032651 25733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:32:31.033152 25733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:32:31.034729 25733 catalog_manager.cc:1347] Generated new cluster ID: ddaa17ab0af14a548098a56514e3f70b
I20260812 16:32:31.034777 25733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:32:31.047897 25733 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:32:31.048136 25733 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:31.056264 25733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd: Generated new TSK 0
I20260812 16:32:31.056486 25733 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:32:31.059506 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:40815
--builtin_ntp_servers=127.19.238.84:41949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 16:32:31.164068 25744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:31.164311 25744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:31.164350 25744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:31.165589 25744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:31.165656 25744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:31.165702 25744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:32:31.167115 25744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:40815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:31.167400 25744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:31.167723 25744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:31.174465 25751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:31.174616 25754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:31.174700 25744 server_base.cc:1034] running on GCE node
W20260812 16:32:31.174487 25750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:31.175063 25744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:31.175354 25744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:31.176517 25744 hybrid_clock.cc:648] HybridClock initialized: now 1786552351176496 us; error 37 us; skew 500 ppm
I20260812 16:32:31.178011 25744 webserver.cc:460] Webserver started at http://127.19.238.65:34567/ using document root <none> and password file <none>
I20260812 16:32:31.178290 25744 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:31.178351 25744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:31.178493 25744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:31.179687 25744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "0acac17b970144dab4c549b53a3d455c"
format_stamp: "Formatted at 2026-08-12 16:32:31 on dist-test-slave-mvvj"
server_key: "3b320c9fc0549d54456d3396eb2540ce"
server_key_iv: "2079f2e5b7a19f2a03997049dfab0f57"
server_key_version: "encryptionkey@0"
I20260812 16:32:31.180138 25744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "0acac17b970144dab4c549b53a3d455c"
format_stamp: "Formatted at 2026-08-12 16:32:31 on dist-test-slave-mvvj"
server_key: "3b320c9fc0549d54456d3396eb2540ce"
server_key_iv: "2079f2e5b7a19f2a03997049dfab0f57"
server_key_version: "encryptionkey@0"
I20260812 16:32:31.181574 25744 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:31.182391 25762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:31.182555 25744 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:31.182664 25744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "0acac17b970144dab4c549b53a3d455c"
format_stamp: "Formatted at 2026-08-12 16:32:31 on dist-test-slave-mvvj"
server_key: "3b320c9fc0549d54456d3396eb2540ce"
server_key_iv: "2079f2e5b7a19f2a03997049dfab0f57"
server_key_version: "encryptionkey@0"
I20260812 16:32:31.182747 25744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:31.186309 25744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:31.186616 25744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:31.186756 25744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:31.187016 25744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:31.187413 25744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:31.187451 25744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:31.187472 25744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:31.187485 25744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:31.195020 25744 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:44783
I20260812 16:32:31.195056 25893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:44783 every 8 connection(s)
I20260812 16:32:31.195449 25744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:32:31.196390 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 25744
I20260812 16:32:31.196525 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:32:31.196754 20409 external_mini_cluster.cc:1428] Setting key 111826b5ea7eb77e6f4719bcc10f6ae4
I20260812 16:32:31.197986 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:40815
--builtin_ntp_servers=127.19.238.84:41949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 16:32:31.199091 25720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.196339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:39031 (local address 127.19.238.126:40815)
0812 16:32:31.196416 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:31.196418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:31.196777 (+ 359us) server_negotiation.cc:407] Connection header received
0812 16:32:31.197462 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:31.197463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:31.197469 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:31.197509 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:31.198843 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:31.198845 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:31.198845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:32:31.198850 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:31.198867 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:32:31.198871 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:31.198931 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:31.199004 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 16:32:31.199019 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 16:32:31.199348 25896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.196342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:40815 (local address 127.19.238.65:39031)
0812 16:32:31.196645 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:31.196690 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:31.197369 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:31.197596 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:31.197600 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:31.197609 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:31.198678 (+ 1069us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:32:31.198679 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:31.198711 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:32:31.198742 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:31.198933 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:31.198939 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:31.198942 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:32:31.199018 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 16:32:31.199093 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":89,"threads_started":1}
I20260812 16:32:31.200011 25894 heartbeater.cc:347] Connected to a master server at 127.19.238.126:40815
I20260812 16:32:31.200126 25894 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:31.200284 25894 heartbeater.cc:511] Master 127.19.238.126:40815 requested a full tablet report, sending...
I20260812 16:32:31.200992 25673 ts_manager.cc:194] Registered new tserver with Master: 0acac17b970144dab4c549b53a3d455c (127.19.238.65:44783)
W20260812 16:32:31.283267 25897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:31.283545 25897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:31.283582 25897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:31.284816 25897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:31.284895 25897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:31.284941 25897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:32:31.286374 25897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:40815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:31.286669 25897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:31.286986 25897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:31.292215 25903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:31.292202 25905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:31.292202 25902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:31.292477 25897 server_base.cc:1034] running on GCE node
I20260812 16:32:31.292685 25897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:31.292891 25897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:31.294030 25897 hybrid_clock.cc:648] HybridClock initialized: now 1786552351294018 us; error 29 us; skew 500 ppm
I20260812 16:32:31.295468 25897 webserver.cc:460] Webserver started at http://127.19.238.66:34437/ using document root <none> and password file <none>
I20260812 16:32:31.295717 25897 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:31.295766 25897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:31.295976 25897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:31.297164 25897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "7a315f8dfbf34f28ab12a54731624cf4"
format_stamp: "Formatted at 2026-08-12 16:32:31 on dist-test-slave-mvvj"
server_key: "44a3bb75d18526a7ad0807e140b40aae"
server_key_iv: "78015ab3f299811064363a9f786fb2f7"
server_key_version: "encryptionkey@0"
I20260812 16:32:31.297593 25897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "7a315f8dfbf34f28ab12a54731624cf4"
format_stamp: "Formatted at 2026-08-12 16:32:31 on dist-test-slave-mvvj"
server_key: "44a3bb75d18526a7ad0807e140b40aae"
server_key_iv: "78015ab3f299811064363a9f786fb2f7"
server_key_version: "encryptionkey@0"
I20260812 16:32:31.299091 25897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:32:31.299894 25911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:31.300076 25897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:31.300179 25897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "7a315f8dfbf34f28ab12a54731624cf4"
format_stamp: "Formatted at 2026-08-12 16:32:31 on dist-test-slave-mvvj"
server_key: "44a3bb75d18526a7ad0807e140b40aae"
server_key_iv: "78015ab3f299811064363a9f786fb2f7"
server_key_version: "encryptionkey@0"
I20260812 16:32:31.300263 25897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:31.303731 25897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:31.304031 25897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:31.304180 25897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:31.304400 25897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:31.304725 25897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:31.304780 25897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:31.304821 25897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:31.304848 25897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:31.310757 25897 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:45005
I20260812 16:32:31.310804 26045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:45005 every 8 connection(s)
I20260812 16:32:31.311193 25897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:32:31.314216 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 25897
I20260812 16:32:31.314333 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:32:31.314579 20409 external_mini_cluster.cc:1428] Setting key 6e89915ffbaf0c8d87222dcb6a9e2084
I20260812 16:32:31.314801 25720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.311968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:33767 (local address 127.19.238.126:40815)
0812 16:32:31.312027 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:31.312029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:31.312336 (+ 307us) server_negotiation.cc:407] Connection header received
0812 16:32:31.312928 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:31.312929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:31.312935 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:31.313000 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:31.314379 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:31.314381 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:31.314382 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:32:31.314387 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:31.314409 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:32:31.314416 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:31.314534 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:31.314672 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 16:32:31.314691 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 16:32:31.314980 26049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.311985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:40815 (local address 127.19.238.66:33767)
0812 16:32:31.312196 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:31.312243 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:31.312799 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:31.313092 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:31.313096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:31.313112 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:31.314159 (+ 1047us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:32:31.314159 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:31.314207 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:32:31.314244 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:31.314551 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:31.314560 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:31.314563 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:32:31.314658 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 16:32:31.314734 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
I20260812 16:32:31.315418 26047 heartbeater.cc:347] Connected to a master server at 127.19.238.126:40815
I20260812 16:32:31.315549 26047 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:31.315716 26047 heartbeater.cc:511] Master 127.19.238.126:40815 requested a full tablet report, sending...
I20260812 16:32:31.315732 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:40815
--builtin_ntp_servers=127.19.238.84:41949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 16:32:31.316256 25673 ts_manager.cc:194] Registered new tserver with Master: 7a315f8dfbf34f28ab12a54731624cf4 (127.19.238.66:45005)
W20260812 16:32:31.401998 26050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:31.402220 26050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:31.402243 26050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:31.403553 26050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:31.403596 26050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:31.403647 26050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:32:31.405114 26050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:40815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:31.405388 26050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:31.405745 26050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:31.411130 26056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:31.411350 26050 server_base.cc:1034] running on GCE node
W20260812 16:32:31.411119 26059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:31.411119 26055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:31.411731 26050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:31.412025 26050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:31.413201 26050 hybrid_clock.cc:648] HybridClock initialized: now 1786552351413183 us; error 29 us; skew 500 ppm
I20260812 16:32:31.414770 26050 webserver.cc:460] Webserver started at http://127.19.238.67:38343/ using document root <none> and password file <none>
I20260812 16:32:31.415023 26050 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:31.415109 26050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:31.415333 26050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:31.416523 26050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "12ecbc2b2dfe4272ab2e2912606125e4"
format_stamp: "Formatted at 2026-08-12 16:32:31 on dist-test-slave-mvvj"
server_key: "ed1775ea6309845253aac7d1aaeb2d4e"
server_key_iv: "78e1422c25a7c2f5731aeb9b9f3095cb"
server_key_version: "encryptionkey@0"
I20260812 16:32:31.416910 26050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "12ecbc2b2dfe4272ab2e2912606125e4"
format_stamp: "Formatted at 2026-08-12 16:32:31 on dist-test-slave-mvvj"
server_key: "ed1775ea6309845253aac7d1aaeb2d4e"
server_key_iv: "78e1422c25a7c2f5731aeb9b9f3095cb"
server_key_version: "encryptionkey@0"
I20260812 16:32:31.418336 26050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:32:31.419143 26067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:31.419319 26050 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:31.419427 26050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "12ecbc2b2dfe4272ab2e2912606125e4"
format_stamp: "Formatted at 2026-08-12 16:32:31 on dist-test-slave-mvvj"
server_key: "ed1775ea6309845253aac7d1aaeb2d4e"
server_key_iv: "78e1422c25a7c2f5731aeb9b9f3095cb"
server_key_version: "encryptionkey@0"
I20260812 16:32:31.419512 26050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:31.422941 26050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:31.423220 26050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:31.423372 26050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:31.423583 26050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:31.423959 26050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:31.424012 26050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:31.424052 26050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:31.424079 26050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:31.429879 26050 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:46679
I20260812 16:32:31.429917 26199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:46679 every 8 connection(s)
I20260812 16:32:31.430260 26050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:32:31.431668 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 26050
I20260812 16:32:31.431788 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:32:31.432010 20409 external_mini_cluster.cc:1428] Setting key c73d5fc04923ae787980edfb80c10764
I20260812 16:32:31.433735 25720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.431015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:57883 (local address 127.19.238.126:40815)
0812 16:32:31.431060 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:31.431063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:31.431290 (+ 227us) server_negotiation.cc:407] Connection header received
0812 16:32:31.431969 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:31.431970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:31.431977 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:31.432041 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:31.433550 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:31.433551 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:31.433551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:32:31.433555 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:31.433569 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:32:31.433573 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:31.433615 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:31.433670 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 16:32:31.433681 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:32:31.433961 26204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.430971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:40815 (local address 127.19.238.67:57883)
0812 16:32:31.431144 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:31.431190 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:31.431863 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:31.432115 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:31.432119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:31.432130 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:31.433408 (+ 1278us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:32:31.433408 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:31.433439 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:32:31.433470 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:31.433621 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:31.433624 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:31.433626 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:32:31.433676 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 16:32:31.433725 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
I20260812 16:32:31.434521 26200 heartbeater.cc:347] Connected to a master server at 127.19.238.126:40815
I20260812 16:32:31.434617 26200 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:31.434733 26200 heartbeater.cc:511] Master 127.19.238.126:40815 requested a full tablet report, sending...
I20260812 16:32:31.435102 25673 ts_manager.cc:194] Registered new tserver with Master: 12ecbc2b2dfe4272ab2e2912606125e4 (127.19.238.67:46679)
I20260812 16:32:31.444486 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:32:31.447222 25720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.446026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42982 (local address 127.19.238.126:40815)
0812 16:32:31.446099 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:31.446101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:31.446247 (+ 146us) server_negotiation.cc:407] Connection header received
0812 16:32:31.446296 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:31.446296 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:31.446302 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:31.446374 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:31.447035 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:31.447036 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:31.447037 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:32:31.447041 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:31.447057 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:32:31.447061 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:31.447110 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:31.447154 (+ 44us) server_negotiation.cc:299] Negotiation successful
0812 16:32:31.447166 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 16:32:31.449003 25673 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:32:31.450652 25673 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:32:31.454936 26211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.453016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:45005 (local address 127.0.0.1:39252)
0812 16:32:31.453319 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:31.453388 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:31.453435 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:31.453738 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:31.453744 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:31.453751 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:31.454262 (+ 511us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:32:31.454263 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:31.454287 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:32:31.454310 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:31.454589 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:31.454597 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:31.454600 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:32:31.454823 (+ 223us) client_negotiation.cc:240] Negotiation successful
0812 16:32:31.454837 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":239,"threads_started":1}
I20260812 16:32:31.455185 26216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.453357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39252 (local address 127.19.238.66:45005)
0812 16:32:31.453563 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:31.453568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:31.453580 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:32:31.453609 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:31.453611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:31.453628 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:31.453710 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:31.454414 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:31.454422 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:31.454422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:32:31.454429 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:31.454466 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:32:31.454488 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:31.454550 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:31.455085 (+ 535us) server_negotiation.cc:299] Negotiation successful
0812 16:32:31.455105 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":83,"threads_started":1}
I20260812 16:32:31.455237 26213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.453109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40142 (local address 127.19.238.65:44783)
0812 16:32:31.454109 (+ 1000us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:31.454113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:31.454129 (+ 16us) server_negotiation.cc:407] Connection header received
0812 16:32:31.454151 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:31.454153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:31.454159 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:31.454243 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:31.454776 (+ 533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:31.454785 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:31.454786 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:32:31.454791 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:31.454830 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:32:31.454861 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:31.455114 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:31.455136 (+ 22us) server_negotiation.cc:299] Negotiation successful
0812 16:32:31.455151 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":961,"thread_start_us":117,"threads_started":1}
I20260812 16:32:31.455049 26212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.453014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:44783 (local address 127.0.0.1:40142)
0812 16:32:31.453320 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:31.453365 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:31.453435 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:31.454263 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:31.454264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:31.454267 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:31.454687 (+ 420us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:32:31.454687 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:31.454693 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:32:31.454704 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:31.454937 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:31.454938 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:31.454938 (+ 0us) client_negotiation.cc:769] Sending connection context
0812 16:32:31.454968 (+ 30us) client_negotiation.cc:240] Negotiation successful
0812 16:32:31.454981 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":62,"threads_started":1}
I20260812 16:32:31.456153 25813 tablet_service.cc:1467] Processing CreateTablet for tablet a221411cc12c4ee4961fb6bbad16e767 (DEFAULT_TABLE table=test-table [id=3bd43f6211884b9db8d28c97c2ce4e64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:31.456151 25968 tablet_service.cc:1467] Processing CreateTablet for tablet a221411cc12c4ee4961fb6bbad16e767 (DEFAULT_TABLE table=test-table [id=3bd43f6211884b9db8d28c97c2ce4e64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:31.456542 25968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a221411cc12c4ee4961fb6bbad16e767. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:31.456542 25813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a221411cc12c4ee4961fb6bbad16e767. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:31.460233 26219 tablet_bootstrap.cc:492] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c: Bootstrap starting.
I20260812 16:32:31.461082 26219 tablet_bootstrap.cc:654] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:31.461442 26219 log.cc:824] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:31.461476 26217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.453440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51950 (local address 127.19.238.67:46679)
0812 16:32:31.453611 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:31.453615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:31.454633 (+ 1018us) server_negotiation.cc:407] Connection header received
0812 16:32:31.454688 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:31.454691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:31.454699 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:31.454799 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:31.460985 (+ 6186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:31.460998 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:31.460998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:32:31.461003 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:31.461045 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:32:31.461082 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:31.461155 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:31.461354 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 16:32:31.461376 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":95,"threads_started":1}
I20260812 16:32:31.461829 26215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.453222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:46679 (local address 127.0.0.1:51950)
0812 16:32:31.454565 (+ 1343us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:31.454576 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:31.454630 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:31.454814 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:31.454815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:31.454817 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:31.460779 (+ 5962us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:32:31.460780 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:31.460789 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:32:31.460827 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:31.461253 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:31.461254 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:31.461255 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 16:32:31.461738 (+ 483us) client_negotiation.cc:240] Negotiation successful
0812 16:32:31.461756 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1325,"thread_start_us":72,"threads_started":1}
I20260812 16:32:31.461815 26218 tablet_bootstrap.cc:492] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4: Bootstrap starting.
I20260812 16:32:31.462567 26219 tablet_bootstrap.cc:492] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c: No bootstrap required, opened a new log
I20260812 16:32:31.462635 26219 ts_tablet_manager.cc:1397] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 16:32:31.462733 26124 tablet_service.cc:1467] Processing CreateTablet for tablet a221411cc12c4ee4961fb6bbad16e767 (DEFAULT_TABLE table=test-table [id=3bd43f6211884b9db8d28c97c2ce4e64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:31.463089 26124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a221411cc12c4ee4961fb6bbad16e767. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:31.465586 26219 raft_consensus.cc:348] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0acac17b970144dab4c549b53a3d455c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44783 } } peers { permanent_uuid: "7a315f8dfbf34f28ab12a54731624cf4" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45005 } } peers { permanent_uuid: "12ecbc2b2dfe4272ab2e2912606125e4" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46679 } }
I20260812 16:32:31.465766 26219 raft_consensus.cc:374] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:31.465803 26219 raft_consensus.cc:729] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0acac17b970144dab4c549b53a3d455c, State: Initialized, Role: FOLLOWER
I20260812 16:32:31.465937 26219 consensus_queue.cc:260] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0acac17b970144dab4c549b53a3d455c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44783 } } peers { permanent_uuid: "7a315f8dfbf34f28ab12a54731624cf4" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45005 } } peers { permanent_uuid: "12ecbc2b2dfe4272ab2e2912606125e4" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46679 } }
I20260812 16:32:31.466277 26219 ts_tablet_manager.cc:1428] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260812 16:32:31.466291 25894 heartbeater.cc:503] Master 127.19.238.126:40815 was elected leader, sending a full tablet report...
I20260812 16:32:31.466678 26223 tablet_bootstrap.cc:492] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4: Bootstrap starting.
I20260812 16:32:31.467101 26218 tablet_bootstrap.cc:654] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:31.467466 26218 log.cc:824] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:31.467725 26223 tablet_bootstrap.cc:654] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:31.468113 26223 log.cc:824] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:31.468358 26218 tablet_bootstrap.cc:492] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4: No bootstrap required, opened a new log
I20260812 16:32:31.468434 26218 ts_tablet_manager.cc:1397] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260812 16:32:31.468914 26223 tablet_bootstrap.cc:492] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4: No bootstrap required, opened a new log
I20260812 16:32:31.468981 26223 ts_tablet_manager.cc:1397] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 16:32:31.471223 26218 raft_consensus.cc:348] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0acac17b970144dab4c549b53a3d455c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44783 } } peers { permanent_uuid: "7a315f8dfbf34f28ab12a54731624cf4" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45005 } } peers { permanent_uuid: "12ecbc2b2dfe4272ab2e2912606125e4" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46679 } }
I20260812 16:32:31.471402 26218 raft_consensus.cc:374] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:31.471439 26218 raft_consensus.cc:729] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a315f8dfbf34f28ab12a54731624cf4, State: Initialized, Role: FOLLOWER
I20260812 16:32:31.471575 26218 consensus_queue.cc:260] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0acac17b970144dab4c549b53a3d455c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44783 } } peers { permanent_uuid: "7a315f8dfbf34f28ab12a54731624cf4" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45005 } } peers { permanent_uuid: "12ecbc2b2dfe4272ab2e2912606125e4" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46679 } }
I20260812 16:32:31.471526 26223 raft_consensus.cc:348] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0acac17b970144dab4c549b53a3d455c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44783 } } peers { permanent_uuid: "7a315f8dfbf34f28ab12a54731624cf4" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45005 } } peers { permanent_uuid: "12ecbc2b2dfe4272ab2e2912606125e4" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46679 } }
I20260812 16:32:31.471661 26223 raft_consensus.cc:374] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:31.471699 26223 raft_consensus.cc:729] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ecbc2b2dfe4272ab2e2912606125e4, State: Initialized, Role: FOLLOWER
I20260812 16:32:31.471880 26218 ts_tablet_manager.cc:1428] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 16:32:31.471820 26223 consensus_queue.cc:260] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0acac17b970144dab4c549b53a3d455c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44783 } } peers { permanent_uuid: "7a315f8dfbf34f28ab12a54731624cf4" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45005 } } peers { permanent_uuid: "12ecbc2b2dfe4272ab2e2912606125e4" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46679 } }
I20260812 16:32:31.471907 26047 heartbeater.cc:503] Master 127.19.238.126:40815 was elected leader, sending a full tablet report...
I20260812 16:32:31.472136 26223 ts_tablet_manager.cc:1428] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 16:32:31.472158 26200 heartbeater.cc:503] Master 127.19.238.126:40815 was elected leader, sending a full tablet report...
I20260812 16:32:31.480995 26222 raft_consensus.cc:482] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:32:31.481122 26222 raft_consensus.cc:504] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0acac17b970144dab4c549b53a3d455c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44783 } } peers { permanent_uuid: "7a315f8dfbf34f28ab12a54731624cf4" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45005 } } peers { permanent_uuid: "12ecbc2b2dfe4272ab2e2912606125e4" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46679 } }
I20260812 16:32:31.481576 26222 leader_election.cc:288] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a315f8dfbf34f28ab12a54731624cf4 (127.19.238.66:45005), 12ecbc2b2dfe4272ab2e2912606125e4 (127.19.238.67:46679)
I20260812 16:32:31.482955 25896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.481754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:46679 (local address 127.19.238.65:42537)
0812 16:32:31.481800 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:31.481813 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:31.481853 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:31.482068 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:31.482069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:31.482072 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:31.482614 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:32:31.482615 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:31.482623 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:32:31.482635 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:31.482813 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:31.482814 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:31.482815 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 16:32:31.482871 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 16:32:31.482888 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 16:32:31.482955 26217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.481828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:42537 (local address 127.19.238.67:46679)
0812 16:32:31.481912 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:31.481914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:31.481922 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:32:31.481954 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:31.481955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:31.481960 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:31.482046 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:31.482716 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:31.482717 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:31.482718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:32:31.482722 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:31.482738 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:32:31.482742 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:31.482815 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:31.482872 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 16:32:31.482887 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 16:32:31.483089 26216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.481825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:44689 (local address 127.19.238.66:45005)
0812 16:32:31.481892 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:31.481895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:31.482177 (+ 282us) server_negotiation.cc:407] Connection header received
0812 16:32:31.482208 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:31.482210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:31.482215 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:31.482278 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:31.482877 (+ 599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:31.482879 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:31.482879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:32:31.482884 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:31.482900 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:32:31.482905 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:31.482953 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:31.483004 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 16:32:31.483019 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 16:32:31.483103 26228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.481763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:45005 (local address 127.19.238.65:44689)
0812 16:32:31.482091 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:31.482101 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:31.482154 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:31.482297 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:31.482298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:31.482300 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:31.482755 (+ 455us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:32:31.482755 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:31.482763 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:32:31.482776 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:31.482960 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:31.482961 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:31.482962 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 16:32:31.483015 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 16:32:31.483028 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":145,"thread_start_us":110,"threads_started":1}
I20260812 16:32:31.483312 26146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a221411cc12c4ee4961fb6bbad16e767" candidate_uuid: "0acac17b970144dab4c549b53a3d455c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ecbc2b2dfe4272ab2e2912606125e4" is_pre_election: true
I20260812 16:32:31.483443 25989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a221411cc12c4ee4961fb6bbad16e767" candidate_uuid: "0acac17b970144dab4c549b53a3d455c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a315f8dfbf34f28ab12a54731624cf4" is_pre_election: true
I20260812 16:32:31.483493 26146 raft_consensus.cc:2454] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0acac17b970144dab4c549b53a3d455c in term 0.
I20260812 16:32:31.483606 25989 raft_consensus.cc:2454] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0acac17b970144dab4c549b53a3d455c in term 0.
I20260812 16:32:31.483762 25765 leader_election.cc:302] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0acac17b970144dab4c549b53a3d455c, 12ecbc2b2dfe4272ab2e2912606125e4; no voters:
I20260812 16:32:31.483901 26222 raft_consensus.cc:2781] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:32:31.483959 26222 raft_consensus.cc:482] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:32:31.483989 26222 raft_consensus.cc:3037] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:31.484699 26222 raft_consensus.cc:504] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0acac17b970144dab4c549b53a3d455c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44783 } } peers { permanent_uuid: "7a315f8dfbf34f28ab12a54731624cf4" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45005 } } peers { permanent_uuid: "12ecbc2b2dfe4272ab2e2912606125e4" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46679 } }
I20260812 16:32:31.484834 26222 leader_election.cc:288] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c [CANDIDATE]: Term 1 election: Requested vote from peers 7a315f8dfbf34f28ab12a54731624cf4 (127.19.238.66:45005), 12ecbc2b2dfe4272ab2e2912606125e4 (127.19.238.67:46679)
I20260812 16:32:31.484983 26146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a221411cc12c4ee4961fb6bbad16e767" candidate_uuid: "0acac17b970144dab4c549b53a3d455c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ecbc2b2dfe4272ab2e2912606125e4"
I20260812 16:32:31.484999 25989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a221411cc12c4ee4961fb6bbad16e767" candidate_uuid: "0acac17b970144dab4c549b53a3d455c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a315f8dfbf34f28ab12a54731624cf4"
I20260812 16:32:31.485075 25989 raft_consensus.cc:3037] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:31.485059 26146 raft_consensus.cc:3037] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:31.485772 26146 raft_consensus.cc:2454] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0acac17b970144dab4c549b53a3d455c in term 1.
I20260812 16:32:31.485947 25765 leader_election.cc:302] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0acac17b970144dab4c549b53a3d455c, 12ecbc2b2dfe4272ab2e2912606125e4; no voters:
I20260812 16:32:31.485940 25989 raft_consensus.cc:2454] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0acac17b970144dab4c549b53a3d455c in term 1.
I20260812 16:32:31.486027 26222 raft_consensus.cc:2781] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:31.486195 26222 raft_consensus.cc:686] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c [term 1 LEADER]: Becoming Leader. State: Replica: 0acac17b970144dab4c549b53a3d455c, State: Running, Role: LEADER
I20260812 16:32:31.486304 26222 consensus_queue.cc:237] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0acac17b970144dab4c549b53a3d455c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44783 } } peers { permanent_uuid: "7a315f8dfbf34f28ab12a54731624cf4" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45005 } } peers { permanent_uuid: "12ecbc2b2dfe4272ab2e2912606125e4" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46679 } }
I20260812 16:32:31.487140 25673 catalog_manager.cc:5495] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0acac17b970144dab4c549b53a3d455c (127.19.238.65). New cstate: current_term: 1 leader_uuid: "0acac17b970144dab4c549b53a3d455c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0acac17b970144dab4c549b53a3d455c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a315f8dfbf34f28ab12a54731624cf4" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12ecbc2b2dfe4272ab2e2912606125e4" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46679 } health_report { overall_health: UNKNOWN } } }
I20260812 16:32:31.490911 26213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.490009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40154 (local address 127.19.238.65:44783)
0812 16:32:31.490059 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:31.490061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:31.490086 (+ 25us) server_negotiation.cc:407] Connection header received
0812 16:32:31.490121 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:31.490122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:31.490136 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:31.490187 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:31.490694 (+ 507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:31.490696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:31.490696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:32:31.490701 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:31.490716 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:32:31.490721 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:31.490773 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:31.490826 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 16:32:31.490840 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:32:31.492700 26146 raft_consensus.cc:1261] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4 [term 1 FOLLOWER]: Refusing update from remote peer 0acac17b970144dab4c549b53a3d455c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:31.493016 25989 raft_consensus.cc:1261] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4 [term 1 FOLLOWER]: Refusing update from remote peer 0acac17b970144dab4c549b53a3d455c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:31.493023 26229 consensus_queue.cc:1035] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c [LEADER]: Connected to new peer: Peer: permanent_uuid: "12ecbc2b2dfe4272ab2e2912606125e4" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:31.493300 26229 consensus_queue.cc:1035] T a221411cc12c4ee4961fb6bbad16e767 P 0acac17b970144dab4c549b53a3d455c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a315f8dfbf34f28ab12a54731624cf4" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:31.497896 25672 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:31.498006 25672 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:31.498756 25672 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26bad0cdb6a04767ae5d48a4610aafcd: Sending DeleteTablet for 3 replicas of tablet a221411cc12c4ee4961fb6bbad16e767
I20260812 16:32:31.499043 25813 tablet_service.cc:1514] Processing DeleteTablet for tablet a221411cc12c4ee4961fb6bbad16e767 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:31 UTC) from {username='slave'} at 127.0.0.1:40142
I20260812 16:32:31.499104 26124 tablet_service.cc:1514] Processing DeleteTablet for tablet a221411cc12c4ee4961fb6bbad16e767 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:31 UTC) from {username='slave'} at 127.0.0.1:51950
I20260812 16:32:31.499315 25968 tablet_service.cc:1514] Processing DeleteTablet for tablet a221411cc12c4ee4961fb6bbad16e767 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:31 UTC) from {username='slave'} at 127.0.0.1:39252
I20260812 16:32:31.499397 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 25744
I20260812 16:32:31.499540 26244 tablet_replica.cc:330] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4: stopping tablet replica
I20260812 16:32:31.499539 26246 tablet_replica.cc:330] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4: stopping tablet replica
I20260812 16:32:31.499642 26244 raft_consensus.cc:2229] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:31.499737 26246 raft_consensus.cc:2229] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:31.499751 26244 raft_consensus.cc:2258] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:31.499948 26246 raft_consensus.cc:2258] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:31.501025 26246 ts_tablet_manager.cc:1905] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:31.501718 26244 ts_tablet_manager.cc:1905] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:31.502657 26246 ts_tablet_manager.cc:1918] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:32:31.502754 26246 log.cc:1196] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/a221411cc12c4ee4961fb6bbad16e767
I20260812 16:32:31.503000 26246 ts_tablet_manager.cc:1939] T a221411cc12c4ee4961fb6bbad16e767 P 7a315f8dfbf34f28ab12a54731624cf4: Deleting consensus metadata
I20260812 16:32:31.503509 25656 catalog_manager.cc:4841] TS 7a315f8dfbf34f28ab12a54731624cf4 (127.19.238.66:45005): tablet a221411cc12c4ee4961fb6bbad16e767 (table test-table [id=3bd43f6211884b9db8d28c97c2ce4e64]) successfully deleted
I20260812 16:32:31.504158 26244 ts_tablet_manager.cc:1918] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:32:31.504240 26244 log.cc:1196] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/a221411cc12c4ee4961fb6bbad16e767
I20260812 16:32:31.504498 26244 ts_tablet_manager.cc:1939] T a221411cc12c4ee4961fb6bbad16e767 P 12ecbc2b2dfe4272ab2e2912606125e4: Deleting consensus metadata
I20260812 16:32:31.504913 25658 catalog_manager.cc:4841] TS 12ecbc2b2dfe4272ab2e2912606125e4 (127.19.238.67:46679): tablet a221411cc12c4ee4961fb6bbad16e767 (table test-table [id=3bd43f6211884b9db8d28c97c2ce4e64]) successfully deleted
W20260812 16:32:31.509711 25658 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.238.65:44783 (error 108)
I20260812 16:32:31.510010 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 25897
I20260812 16:32:31.510272 26215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.510120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:44783 (local address 127.0.0.1:40156)
0812 16:32:31.510185 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:31.510228 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:44783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 16:32:31.511435 25658 catalog_manager.cc:4584] TS 0acac17b970144dab4c549b53a3d455c (127.19.238.65:44783): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a221411cc12c4ee4961fb6bbad16e767: Network error: Client connection negotiation failed: client connection to 127.19.238.65:44783: connect: Connection refused (error 111)
I20260812 16:32:31.518447 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 26050
I20260812 16:32:31.525753 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 25639
2026-08-12T16:32:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (649 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 16:32:31.547327 20409 mini_oidc.cc:152] Starting JWKS server
I20260812 16:32:31.548624 20409 webserver.cc:460] Webserver started at https://127.0.0.1:40469/ using document root <none> and password file <none>
I20260812 16:32:31.548736 20409 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 16:32:31.548911 20409 webserver.cc:460] Webserver started at http://127.0.0.1:44435/ using document root <none> and password file <none>
2026-08-12T16:32:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:32:31Z Disabled control of system clock
I20260812 16:32:31.557934 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:40703
--webserver_interface=127.19.238.126
--webserver_port=0
--builtin_ntp_servers=127.19.238.84:32995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:40703
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40469/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0/testchainca.cert with env {}
W20260812 16:32:31.645236 26254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:31.645498 26254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:31.645529 26254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:31.646751 26254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:31.646792 26254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:31.646840 26254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:31.646865 26254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:31.646878 26254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 16:32:31.646908 26254 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 16:32:31.646919 26254 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40469/jwks.json
I20260812 16:32:31.648595 26254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:32995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:40703
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:40703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40469/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:31.648819 26254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:31.649009 26254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:31.654206 26264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:31.654225 26261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:31.654381 26254 server_base.cc:1034] running on GCE node
W20260812 16:32:31.654222 26260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:31.654767 26254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:31.655081 26254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:31.656273 26254 hybrid_clock.cc:648] HybridClock initialized: now 1786552351656259 us; error 31 us; skew 500 ppm
I20260812 16:32:31.657732 26254 webserver.cc:460] Webserver started at http://127.19.238.126:38129/ using document root <none> and password file <none>
I20260812 16:32:31.657997 26254 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:31.658071 26254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:31.658231 26254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:31.659395 26254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "5c267f727bd2462881e282916bfc6f2c"
format_stamp: "Formatted at 2026-08-12 16:32:31 on dist-test-slave-mvvj"
server_key: "9c360ffa2e84c20f1b98c9c8d732a4e8"
server_key_iv: "03d57a0306a19464d424aa1fa7da680b"
server_key_version: "encryptionkey@0"
I20260812 16:32:31.659765 26254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "5c267f727bd2462881e282916bfc6f2c"
format_stamp: "Formatted at 2026-08-12 16:32:31 on dist-test-slave-mvvj"
server_key: "9c360ffa2e84c20f1b98c9c8d732a4e8"
server_key_iv: "03d57a0306a19464d424aa1fa7da680b"
server_key_version: "encryptionkey@0"
I20260812 16:32:31.661158 26254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:31.662007 26271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:31.662210 26254 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:31.662343 26254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "5c267f727bd2462881e282916bfc6f2c"
format_stamp: "Formatted at 2026-08-12 16:32:31 on dist-test-slave-mvvj"
server_key: "9c360ffa2e84c20f1b98c9c8d732a4e8"
server_key_iv: "03d57a0306a19464d424aa1fa7da680b"
server_key_version: "encryptionkey@0"
I20260812 16:32:31.662431 26254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:31.675130 26254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:31.675514 26254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:31.675702 26254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:31.680303 26254 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:40703
I20260812 16:32:31.680338 26332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:40703 every 8 connection(s)
I20260812 16:32:31.680748 26254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:32:31.681804 26333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:31.683314 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 26254
I20260812 16:32:31.683430 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:32:31.683626 20409 external_mini_cluster.cc:1428] Setting key b61c25d004aee82531b2e3e2fd188ec2
I20260812 16:32:31.684828 26333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c: Bootstrap starting.
I20260812 16:32:31.685563 26333 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:31.685892 26333 log.cc:824] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:31.686591 26333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c: No bootstrap required, opened a new log
I20260812 16:32:31.688478 26333 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c267f727bd2462881e282916bfc6f2c" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40703 } }
I20260812 16:32:31.688656 26333 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:31.688707 26333 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c267f727bd2462881e282916bfc6f2c, State: Initialized, Role: FOLLOWER
I20260812 16:32:31.688836 26333 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c267f727bd2462881e282916bfc6f2c" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40703 } }
I20260812 16:32:31.688906 26333 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:32:31.688933 26333 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:32:31.688978 26333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:31.689541 26339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.684237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44312 (local address 127.19.238.126:40703)
0812 16:32:31.684464 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:31.684472 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:31.684498 (+ 26us) server_negotiation.cc:407] Connection header received
0812 16:32:31.685004 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:31.685012 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:31.685096 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:31.685223 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:31.685456 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:31.686669 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:31.687519 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:31.687960 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:31.688102 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:31.688830 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:31.688834 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:31.688835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:32:31.689062 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:31.689111 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:32:31.689122 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:31.689192 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:31.689269 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 16:32:31.689348 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260812 16:32:31.689685 26333 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c267f727bd2462881e282916bfc6f2c" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40703 } }
I20260812 16:32:31.689802 26333 leader_election.cc:302] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c267f727bd2462881e282916bfc6f2c; no voters:
I20260812 16:32:31.689970 26333 leader_election.cc:288] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:32:31.690057 26341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:31.690230 26341 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c [term 1 LEADER]: Becoming Leader. State: Replica: 5c267f727bd2462881e282916bfc6f2c, State: Running, Role: LEADER
I20260812 16:32:31.690271 26333 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:31.690421 26341 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c267f727bd2462881e282916bfc6f2c" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40703 } }
I20260812 16:32:31.690495 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:32:31.690835 26343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c267f727bd2462881e282916bfc6f2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c267f727bd2462881e282916bfc6f2c" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40703 } } }
I20260812 16:32:31.690814 26341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c267f727bd2462881e282916bfc6f2c. Latest consensus state: current_term: 1 leader_uuid: "5c267f727bd2462881e282916bfc6f2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c267f727bd2462881e282916bfc6f2c" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 40703 } } }
I20260812 16:32:31.690907 26343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:31.690914 26341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:31.691753 26349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:32:31.692263 26349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:32:31.693733 26349 catalog_manager.cc:1347] Generated new cluster ID: 2430608aac3e427cbee168cbc4b11254
I20260812 16:32:31.693787 26349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:32:31.698853 26349 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:32:31.699565 26349 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:31.707146 26349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c267f727bd2462881e282916bfc6f2c: Generated new TSK 0
I20260812 16:32:31.707371 26349 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 16:32:31.732455 26370 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 16:32:31.732632 26370 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 16:32:31.732883 26339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.716480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44326 (local address 127.19.238.126:40703)
0812 16:32:31.716533 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:31.716535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:31.716671 (+ 136us) server_negotiation.cc:407] Connection header received
0812 16:32:31.716766 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:31.716767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:31.716780 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:31.716852 (+ 72us) server_negotiation.cc:226] Negotiated authn=JWT
0812 16:32:31.717001 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:31.717857 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:31.718704 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:31.719022 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:31.719095 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:31.719347 (+ 252us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 16:32:31.732627 (+ 13280us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40469/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 16:32:31.732803 (+ 176us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44326: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40469/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 16:32:31.732969 26339 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44326: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40469/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 16:32:31.733475 26369 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:31.716295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:40703 (local address 127.0.0.1:44326)
0812 16:32:31.716591 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:31.716602 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:31.716671 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:31.716874 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:31.716875 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:31.716877 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 16:32:31.716934 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:31.716936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:31.717977 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:31.717978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:31.718615 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:31.718618 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:31.719144 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:31.719145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:31.719203 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:31.719224 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 16:32:31.732843 (+ 13619us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40469/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 16:32:31.732983 (+ 140us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.238.126:40703: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40469/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":140,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.19.238.126:40703: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40469/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 16:32:31.734023 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 26254
2026-08-12T16:32:31Z chronyd exiting
I20260812 16:32:31.775126 20409 test_util.cc:175] -----------------------------------------------
I20260812 16:32:31.775216 20409 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786552327177731-20409-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (228 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 16:32:31.776067 20409 mini_oidc.cc:152] Starting JWKS server
I20260812 16:32:31.776775 20409 webserver.cc:460] Webserver started at https://127.0.0.1:45461/ using document root <none> and password file <none>
I20260812 16:32:31.776849 20409 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 16:32:31.776999 20409 webserver.cc:460] Webserver started at http://127.0.0.1:33233/ using document root <none> and password file <none>
2026-08-12T16:32:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:32:31Z Disabled control of system clock
I20260812 16:32:31.786401 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:44821
--webserver_interface=127.19.238.126
--webserver_port=0
--builtin_ntp_servers=127.19.238.84:35755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:44821
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45461/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552327177731-20409-0/testchainca.cert with env {}
W20260812 16:32:31.870116 26377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:31.870435 26377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:31.870483 26377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:31.871757 26377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:31.871802 26377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:31.871815 26377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:31.871856 26377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:31.871869 26377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 16:32:31.871882 26377 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 16:32:31.871893 26377 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45461/jwks.json
I20260812 16:32:31.873413 26377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:35755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:44821
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:44821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45461/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552327177731-20409-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:31.873626 26377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:31.873797 26377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:31.878767 26387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:31.878708 26383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:31.878710 26384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:31.879580 26377 server_base.cc:1034] running on GCE node
I20260812 16:32:31.879896 26377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:31.880211 26377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:31.881402 26377 hybrid_clock.cc:648] HybridClock initialized: now 1786552351881376 us; error 29 us; skew 500 ppm
I20260812 16:32:31.882863 26377 webserver.cc:460] Webserver started at http://127.19.238.126:44757/ using document root <none> and password file <none>
I20260812 16:32:31.883148 26377 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:31.883201 26377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:31.883378 26377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:31.884621 26377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "9311158747ab440788634c41e72b3fc6"
format_stamp: "Formatted at 2026-08-12 16:32:31 on dist-test-slave-mvvj"
server_key: "5e0f2ebfe0b551f431cb35d357b7cb94"
server_key_iv: "dcdc036f0c4394d3e55b056aa0f67a5e"
server_key_version: "encryptionkey@0"
I20260812 16:32:31.884994 26377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "9311158747ab440788634c41e72b3fc6"
format_stamp: "Formatted at 2026-08-12 16:32:31 on dist-test-slave-mvvj"
server_key: "5e0f2ebfe0b551f431cb35d357b7cb94"
server_key_iv: "dcdc036f0c4394d3e55b056aa0f67a5e"
server_key_version: "encryptionkey@0"
I20260812 16:32:31.886322 26377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:32:31.887055 26393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:31.887246 26377 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:31.887354 26377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "9311158747ab440788634c41e72b3fc6"
format_stamp: "Formatted at 2026-08-12 16:32:31 on dist-test-slave-mvvj"
server_key: "5e0f2ebfe0b551f431cb35d357b7cb94"
server_key_iv: "dcdc036f0c4394d3e55b056aa0f67a5e"
server_key_version: "encryptionkey@0"
I20260812 16:32:31.887441 26377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:31.904429 26377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:31.904811 26377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:31.904980 26377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:31.909226 26377 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:44821
I20260812 16:32:31.909324 26454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:44821 every 8 connection(s)
I20260812 16:32:31.909680 26377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:32:31.910455 26455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:31.911543 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 26377
I20260812 16:32:31.911659 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:32:31.911890 20409 external_mini_cluster.cc:1428] Setting key 74250495ca9f7bde1be11ff97d9de1be
I20260812 16:32:31.914161 26455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6: Bootstrap starting.
I20260812 16:32:31.914898 26455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:31.915221 26455 log.cc:824] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:31.916092 26455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6: No bootstrap required, opened a new log
I20260812 16:32:31.918088 26458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.912507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52246 (local address 127.19.238.126:44821)
0812 16:32:31.912720 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:31.912728 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:31.912754 (+ 26us) server_negotiation.cc:407] Connection header received
0812 16:32:31.913194 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:31.913201 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:31.913263 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:31.913386 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:31.913622 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:31.914577 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:31.915767 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:31.916415 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:31.916556 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:31.917339 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:31.917343 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:31.917344 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:32:31.917557 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:31.917602 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:32:31.917617 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:31.917691 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:31.917765 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 16:32:31.917868 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260812 16:32:31.918726 26455 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9311158747ab440788634c41e72b3fc6" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 44821 } }
I20260812 16:32:31.918943 26455 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:31.918993 26455 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9311158747ab440788634c41e72b3fc6, State: Initialized, Role: FOLLOWER
I20260812 16:32:31.919104 26455 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9311158747ab440788634c41e72b3fc6" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 44821 } }
I20260812 16:32:31.919193 26455 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:32:31.919173 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:32:31.919237 26455 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:32:31.919287 26455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:31.920053 26455 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9311158747ab440788634c41e72b3fc6" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 44821 } }
I20260812 16:32:31.920162 26455 leader_election.cc:302] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9311158747ab440788634c41e72b3fc6; no voters:
I20260812 16:32:31.920365 26455 leader_election.cc:288] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:32:31.920419 26460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:31.920595 26460 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6 [term 1 LEADER]: Becoming Leader. State: Replica: 9311158747ab440788634c41e72b3fc6, State: Running, Role: LEADER
I20260812 16:32:31.920641 26455 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:31.920683 26460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9311158747ab440788634c41e72b3fc6" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 44821 } }
I20260812 16:32:31.921131 26461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9311158747ab440788634c41e72b3fc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9311158747ab440788634c41e72b3fc6" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 44821 } } }
I20260812 16:32:31.921233 26461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:31.921415 26462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9311158747ab440788634c41e72b3fc6. Latest consensus state: current_term: 1 leader_uuid: "9311158747ab440788634c41e72b3fc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9311158747ab440788634c41e72b3fc6" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 44821 } } }
I20260812 16:32:31.921448 26471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:32:31.921984 26471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:32:31.922616 26462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:31.923640 26471 catalog_manager.cc:1347] Generated new cluster ID: 3078c8578dd1453eb3d5f220701fc492
I20260812 16:32:31.923710 26471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:32:31.942823 26471 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:32:31.943631 26471 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:31.947255 26471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9311158747ab440788634c41e72b3fc6: Generated new TSK 0
I20260812 16:32:31.947465 26471 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 16:32:31.975172 26491 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 16:32:31.975281 26491 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 16:32:31.975627 26458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:31.959913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52254 (local address 127.19.238.126:44821)
0812 16:32:31.959968 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:31.959971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:31.960125 (+ 154us) server_negotiation.cc:407] Connection header received
0812 16:32:31.960211 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:31.960212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:31.960224 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:31.960291 (+ 67us) server_negotiation.cc:226] Negotiated authn=JWT
0812 16:32:31.960440 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:31.961008 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:31.962173 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:31.962385 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:31.962461 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:31.962713 (+ 252us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 16:32:31.975420 (+ 12707us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45461/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 16:32:31.975569 (+ 149us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52254: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45461/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":27}
W20260812 16:32:31.975710 26458 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52254: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45461/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 16:32:31.975857 26490 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:31.959846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:44821 (local address 127.0.0.1:52254)
0812 16:32:31.960039 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:31.960053 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:31.960131 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:31.960308 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:31.960308 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:31.960310 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 16:32:31.960373 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:31.960376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:31.961095 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:31.961096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:31.962060 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:31.962064 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:31.962477 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:31.962478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:31.962551 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:31.962558 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 16:32:31.975571 (+ 13013us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45461/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 16:32:31.975722 (+ 151us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.238.126:44821: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45461/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":106,"threads_started":1}
I20260812 16:32:31.976240 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 26377
2026-08-12T16:32:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (243 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 16:32:32.019640 20409 mini_oidc.cc:152] Starting JWKS server
I20260812 16:32:32.020434 20409 webserver.cc:460] Webserver started at https://127.0.0.1:41313/ using document root <none> and password file <none>
I20260812 16:32:32.020507 20409 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 16:32:32.020669 20409 webserver.cc:460] Webserver started at http://127.0.0.1:40991/ using document root <none> and password file <none>
2026-08-12T16:32:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:32:32Z Disabled control of system clock
I20260812 16:32:32.030097 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:43095
--webserver_interface=127.19.238.126
--webserver_port=0
--builtin_ntp_servers=127.19.238.84:32845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:43095
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41313/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 16:32:32.116065 26497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:32.116292 26497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:32.116317 26497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:32.117538 26497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:32.117585 26497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:32.117599 26497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:32.117610 26497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:32.117623 26497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 16:32:32.117635 26497 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 16:32:32.117686 26497 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41313/jwks.json
I20260812 16:32:32.119136 26497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:32845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:43095
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:43095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41313/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:32.119437 26497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:32.119757 26497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:32.124775 26507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:32.124758 26505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:32.124818 26503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:32.125069 26497 server_base.cc:1034] running on GCE node
I20260812 16:32:32.125344 26497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:32.125654 26497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:32.126847 26497 hybrid_clock.cc:648] HybridClock initialized: now 1786552352126818 us; error 43 us; skew 500 ppm
I20260812 16:32:32.128206 26497 webserver.cc:460] Webserver started at http://127.19.238.126:43425/ using document root <none> and password file <none>
I20260812 16:32:32.128464 26497 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:32.128536 26497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:32.128701 26497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:32.129817 26497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "15f821bd58d24b4b82f29a2479d7b633"
format_stamp: "Formatted at 2026-08-12 16:32:32 on dist-test-slave-mvvj"
server_key: "4e760255e78d3f6b2a73097979434cb5"
server_key_iv: "f9dc6b3ffd8fa10b7cb4198b54dcbbde"
server_key_version: "encryptionkey@0"
I20260812 16:32:32.130200 26497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "15f821bd58d24b4b82f29a2479d7b633"
format_stamp: "Formatted at 2026-08-12 16:32:32 on dist-test-slave-mvvj"
server_key: "4e760255e78d3f6b2a73097979434cb5"
server_key_iv: "f9dc6b3ffd8fa10b7cb4198b54dcbbde"
server_key_version: "encryptionkey@0"
I20260812 16:32:32.131654 26497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:32:32.132581 26514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:32.132756 26497 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:32.132859 26497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "15f821bd58d24b4b82f29a2479d7b633"
format_stamp: "Formatted at 2026-08-12 16:32:32 on dist-test-slave-mvvj"
server_key: "4e760255e78d3f6b2a73097979434cb5"
server_key_iv: "f9dc6b3ffd8fa10b7cb4198b54dcbbde"
server_key_version: "encryptionkey@0"
I20260812 16:32:32.132946 26497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:32.144752 26497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:32.145138 26497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:32.145309 26497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:32.149664 26497 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:43095
I20260812 16:32:32.149725 26575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:43095 every 8 connection(s)
I20260812 16:32:32.150113 26497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:32:32.150902 26576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:32.154006 26576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633: Bootstrap starting.
I20260812 16:32:32.154673 26576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:32.154999 26576 log.cc:824] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:32.155019 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 26497
I20260812 16:32:32.155143 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:32:32.155339 20409 external_mini_cluster.cc:1428] Setting key 645c287fcda71541005923535369669f
I20260812 16:32:32.156322 26576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633: No bootstrap required, opened a new log
I20260812 16:32:32.159260 26576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f821bd58d24b4b82f29a2479d7b633" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 43095 } }
I20260812 16:32:32.159427 26576 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:32.159456 26576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f821bd58d24b4b82f29a2479d7b633, State: Initialized, Role: FOLLOWER
I20260812 16:32:32.159608 26576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f821bd58d24b4b82f29a2479d7b633" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 43095 } }
I20260812 16:32:32.159716 26576 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:32:32.159762 26576 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:32:32.159808 26576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:32.160655 26576 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f821bd58d24b4b82f29a2479d7b633" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 43095 } }
I20260812 16:32:32.160784 26576 leader_election.cc:302] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15f821bd58d24b4b82f29a2479d7b633; no voters:
I20260812 16:32:32.160981 26576 leader_election.cc:288] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:32:32.161058 26583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:32.161229 26583 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633 [term 1 LEADER]: Becoming Leader. State: Replica: 15f821bd58d24b4b82f29a2479d7b633, State: Running, Role: LEADER
I20260812 16:32:32.161266 26576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:32.161310 26583 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f821bd58d24b4b82f29a2479d7b633" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 43095 } }
I20260812 16:32:32.161669 26584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15f821bd58d24b4b82f29a2479d7b633" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f821bd58d24b4b82f29a2479d7b633" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 43095 } } }
I20260812 16:32:32.161757 26584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:32.161938 26585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15f821bd58d24b4b82f29a2479d7b633. Latest consensus state: current_term: 1 leader_uuid: "15f821bd58d24b4b82f29a2479d7b633" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f821bd58d24b4b82f29a2479d7b633" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 43095 } } }
I20260812 16:32:32.162010 26585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:32.162490 26590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:32:32.163043 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:32.155899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46852 (local address 127.19.238.126:43095)
0812 16:32:32.156145 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:32.156156 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:32.156191 (+ 35us) server_negotiation.cc:407] Connection header received
0812 16:32:32.156837 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:32.156846 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:32.156948 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:32.157122 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:32.157335 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:32.158978 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:32.160003 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:32.160702 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:32.160844 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:32.161648 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:32.161653 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:32.161654 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:32:32.162088 (+ 434us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:32.162162 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:32:32.162180 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:32.162612 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:32.162656 (+ 44us) server_negotiation.cc:299] Negotiation successful
0812 16:32:32.162763 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260812 16:32:32.163055 26590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:32:32.164381 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:32:32.164938 26590 catalog_manager.cc:1347] Generated new cluster ID: c14a1fe9d236470181f8c58c3c767766
I20260812 16:32:32.164996 26590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:32:32.175251 26590 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:32:32.175908 26590 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:32.184640 26590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15f821bd58d24b4b82f29a2479d7b633: Generated new TSK 0
I20260812 16:32:32.184901 26590 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:32:32.198669 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:32.194521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46858 (local address 127.19.238.126:43095)
0812 16:32:32.194597 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:32.194600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:32.194722 (+ 122us) server_negotiation.cc:407] Connection header received
0812 16:32:32.194805 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:32.194806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:32.194821 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:32.194895 (+ 74us) server_negotiation.cc:226] Negotiated authn=JWT
0812 16:32:32.195125 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:32.195721 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:32.196943 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:32.197140 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:32.197200 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:32.197470 (+ 270us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 16:32:32.198424 (+ 954us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41313/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 16:32:32.198586 (+ 162us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46858: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41313/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 16:32:32.198745 26611 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:32.194426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:43095 (local address 127.0.0.1:46858)
0812 16:32:32.194637 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:32.194650 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:32.194727 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:32.194918 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:32.194919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:32.194921 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 16:32:32.195018 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:32.195022 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:32.195823 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:32.195824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:32.196845 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:32.196849 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:32.197235 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:32.197236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:32.197302 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:32.197308 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 16:32:32.198536 (+ 1228us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41313/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 16:32:32.198638 (+ 102us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.238.126:43095: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41313/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":142,"threads_started":1}
W20260812 16:32:32.198751 26581 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46858: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41313/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260812 16:32:32.199462 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 26497
E20260812 16:32:32.210906 26612 webserver.cc:533] Webserver: sslize failed:
E20260812 16:32:32.210999 26612 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T16:32:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (220 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T16:32:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:32:32Z Disabled control of system clock
I20260812 16:32:32.249897 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:35853
--webserver_interface=127.19.238.126
--webserver_port=0
--builtin_ntp_servers=127.19.238.84:36889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:35853
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552327177731-20409-0/testchainca.cert with env {}
W20260812 16:32:32.332973 26616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:32.333194 26616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:32.333216 26616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:32.334555 26616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:32.334600 26616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:32.334613 26616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:32.334626 26616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:32.334676 26616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 16:32:32.334694 26616 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 16:32:32.334718 26616 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 16:32:32.336323 26616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:36889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:35853
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:35853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552327177731-20409-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:32.336593 26616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:32.336824 26616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:32.342103 26622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:32.342105 26623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:32.342103 26627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:32.342341 26616 server_base.cc:1034] running on GCE node
I20260812 16:32:32.342644 26616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:32.342900 26616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:32.344043 26616 hybrid_clock.cc:648] HybridClock initialized: now 1786552352344020 us; error 37 us; skew 500 ppm
I20260812 16:32:32.345453 26616 webserver.cc:460] Webserver started at http://127.19.238.126:45953/ using document root <none> and password file <none>
I20260812 16:32:32.345726 26616 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:32.345774 26616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:32.345963 26616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:32.347105 26616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "a380f306c0d94ed2b5104758cb13bfc9"
format_stamp: "Formatted at 2026-08-12 16:32:32 on dist-test-slave-mvvj"
server_key: "6f5a213d4bd89c391bbf981302856d1c"
server_key_iv: "d8a774ea2b1e23aa632f114e6b2c5863"
server_key_version: "encryptionkey@0"
I20260812 16:32:32.347548 26616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "a380f306c0d94ed2b5104758cb13bfc9"
format_stamp: "Formatted at 2026-08-12 16:32:32 on dist-test-slave-mvvj"
server_key: "6f5a213d4bd89c391bbf981302856d1c"
server_key_iv: "d8a774ea2b1e23aa632f114e6b2c5863"
server_key_version: "encryptionkey@0"
I20260812 16:32:32.349062 26616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:32:32.349841 26635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:32.350042 26616 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:32.350136 26616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "a380f306c0d94ed2b5104758cb13bfc9"
format_stamp: "Formatted at 2026-08-12 16:32:32 on dist-test-slave-mvvj"
server_key: "6f5a213d4bd89c391bbf981302856d1c"
server_key_iv: "d8a774ea2b1e23aa632f114e6b2c5863"
server_key_version: "encryptionkey@0"
I20260812 16:32:32.350215 26616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:32.368719 26616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:32.369079 26616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:32.369239 26616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:32.373667 26616 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:35853
I20260812 16:32:32.373704 26695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:35853 every 8 connection(s)
I20260812 16:32:32.374075 26616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:32:32.375049 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 26616
I20260812 16:32:32.375154 26696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:32.375188 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:32:32.375422 20409 external_mini_cluster.cc:1428] Setting key 45700b1761f2b6133195b23928af4736
I20260812 16:32:32.378944 26696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9: Bootstrap starting.
I20260812 16:32:32.379627 26696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:32.379995 26696 log.cc:824] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:32.380878 26696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9: No bootstrap required, opened a new log
I20260812 16:32:32.382230 26700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:32.376120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53266 (local address 127.19.238.126:35853)
0812 16:32:32.376341 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:32.376350 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:32.376391 (+ 41us) server_negotiation.cc:407] Connection header received
0812 16:32:32.376970 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:32.376980 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:32.377071 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:32.377226 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:32.377536 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:32.378922 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:32.380021 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:32.380537 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:32.380677 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:32.381477 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:32.381482 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:32.381482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:32:32.381707 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:32.381763 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:32:32.381780 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:32.381851 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:32.381932 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 16:32:32.382017 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":80,"threads_started":1}
I20260812 16:32:32.383595 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:32:32.383513 26696 raft_consensus.cc:348] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a380f306c0d94ed2b5104758cb13bfc9" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 35853 } }
I20260812 16:32:32.383715 26696 raft_consensus.cc:374] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:32.383764 26696 raft_consensus.cc:729] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a380f306c0d94ed2b5104758cb13bfc9, State: Initialized, Role: FOLLOWER
I20260812 16:32:32.383913 26696 consensus_queue.cc:260] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a380f306c0d94ed2b5104758cb13bfc9" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 35853 } }
I20260812 16:32:32.383996 26696 raft_consensus.cc:388] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:32:32.384037 26696 raft_consensus.cc:482] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:32:32.384097 26696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:32.384938 26696 raft_consensus.cc:504] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a380f306c0d94ed2b5104758cb13bfc9" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 35853 } }
I20260812 16:32:32.385062 26696 leader_election.cc:302] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a380f306c0d94ed2b5104758cb13bfc9; no voters:
I20260812 16:32:32.385241 26696 leader_election.cc:288] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:32:32.385274 26702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:32.385502 26702 raft_consensus.cc:686] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9 [term 1 LEADER]: Becoming Leader. State: Replica: a380f306c0d94ed2b5104758cb13bfc9, State: Running, Role: LEADER
I20260812 16:32:32.385514 26696 sys_catalog.cc:564] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:32.385607 26702 consensus_queue.cc:237] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a380f306c0d94ed2b5104758cb13bfc9" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 35853 } }
I20260812 16:32:32.386023 26704 sys_catalog.cc:455] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a380f306c0d94ed2b5104758cb13bfc9. Latest consensus state: current_term: 1 leader_uuid: "a380f306c0d94ed2b5104758cb13bfc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a380f306c0d94ed2b5104758cb13bfc9" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 35853 } } }
I20260812 16:32:32.386003 26703 sys_catalog.cc:455] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a380f306c0d94ed2b5104758cb13bfc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a380f306c0d94ed2b5104758cb13bfc9" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 35853 } } }
I20260812 16:32:32.386153 26704 sys_catalog.cc:458] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:32.386162 26703 sys_catalog.cc:458] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:32.386739 26708 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:32:32.387362 26708 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:32:32.388886 26708 catalog_manager.cc:1347] Generated new cluster ID: 36c4e0b973dc4d0198cb10d951e70cf3
I20260812 16:32:32.388949 26708 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:32:32.416493 26708 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:32:32.417080 26708 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:32.423008 26708 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a380f306c0d94ed2b5104758cb13bfc9: Generated new TSK 0
I20260812 16:32:32.423241 26708 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:32:32.908815 26700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:32.430493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53276 (local address 127.19.238.126:35853)
0812 16:32:32.430560 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:32.430563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:32.430606 (+ 43us) server_negotiation.cc:407] Connection header received
0812 16:32:32.430682 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:32.430684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:32.430700 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:32.430775 (+ 75us) server_negotiation.cc:226] Negotiated authn=JWT
0812 16:32:32.430909 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:32.431685 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:32.432549 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:32.432870 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:32.432944 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:32.433106 (+ 162us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 16:32:32.908564 (+475458us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 16:32:32.908748 (+ 184us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53276: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":29}
W20260812 16:32:32.908936 26700 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53276: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 16:32:32.908963 26731 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:32.430376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:35853 (local address 127.0.0.1:53276)
0812 16:32:32.430545 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:32.430555 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:32.430613 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:32.430780 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:32.430781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:32.430783 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 16:32:32.430837 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:32.430839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:32.431797 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:32.431798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:32.432448 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:32.432451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:32.432954 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:32.432955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:32.433000 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:32.433004 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 16:32:32.908732 (+475728us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 16:32:32.908873 (+ 141us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.238.126:35853: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":92,"threads_started":1}
I20260812 16:32:32.909437 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 26616
2026-08-12T16:32:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (689 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (93 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (94 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:32:33 dist-test-slave-mvvj krb5kdc[26745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:32:33 dist-test-slave-mvvj krb5kdc[26745](info): set up 2 sockets
Aug 12 16:32:33 dist-test-slave-mvvj krb5kdc[26745](info): commencing operation
krb5kdc: starting...
W20260812 16:32:34.223978 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:32:34 dist-test-slave-mvvj krb5kdc[26745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:32:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:32:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:32:34.355067 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:42385
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:37437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:42385
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:34.440369 26761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:34.440634 26761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:34.440676 26761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:34.441908 26761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:34.441988 26761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:34.442013 26761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:34.442047 26761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:34.442075 26761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:32:34.443655 26761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:37437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:42385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:42385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:34.443986 26761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:34.444245 26761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:34.449396 26767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:34.449386 26768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:34.449386 26770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:34.449523 26761 server_base.cc:1034] running on GCE node
I20260812 16:32:34.450060 26761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:34.450393 26761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:34.451570 26761 hybrid_clock.cc:648] HybridClock initialized: now 1786552354451570 us; error 44 us; skew 500 ppm
Aug 12 16:32:34 dist-test-slave-mvvj krb5kdc[26745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552354, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:34.454420 26761 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:32:34.454972 26761 webserver.cc:460] Webserver started at http://127.19.238.126:41635/ using document root <none> and password file <none>
I20260812 16:32:34.455231 26761 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:34.455304 26761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:34.455448 26761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:34.456657 26761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "879084ca35cb428594d1d7804540d0c8"
format_stamp: "Formatted at 2026-08-12 16:32:34 on dist-test-slave-mvvj"
server_key: "7020a91eb520c9899f79521172cdcb6b"
server_key_iv: "2d0c6fe92df985d6587781a9462e818d"
server_key_version: "encryptionkey@0"
I20260812 16:32:34.457052 26761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "879084ca35cb428594d1d7804540d0c8"
format_stamp: "Formatted at 2026-08-12 16:32:34 on dist-test-slave-mvvj"
server_key: "7020a91eb520c9899f79521172cdcb6b"
server_key_iv: "2d0c6fe92df985d6587781a9462e818d"
server_key_version: "encryptionkey@0"
I20260812 16:32:34.458499 26761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:32:34.459296 26778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:34.459518 26761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:34.459632 26761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "879084ca35cb428594d1d7804540d0c8"
format_stamp: "Formatted at 2026-08-12 16:32:34 on dist-test-slave-mvvj"
server_key: "7020a91eb520c9899f79521172cdcb6b"
server_key_iv: "2d0c6fe92df985d6587781a9462e818d"
server_key_version: "encryptionkey@0"
I20260812 16:32:34.459712 26761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:34.475386 26761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:34.477784 26761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:34.477965 26761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:34.482858 26761 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:42385
I20260812 16:32:34.482944 26840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:42385 every 8 connection(s)
I20260812 16:32:34.483343 26761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:32:34.484314 26841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:34.486828 26841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8: Bootstrap starting.
I20260812 16:32:34.487433 26841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:34.487702 26841 log.cc:824] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:34.488574 26841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8: No bootstrap required, opened a new log
I20260812 16:32:34.490198 26841 raft_consensus.cc:348] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879084ca35cb428594d1d7804540d0c8" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42385 } }
I20260812 16:32:34.490356 26841 raft_consensus.cc:374] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:34.490389 26841 raft_consensus.cc:729] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879084ca35cb428594d1d7804540d0c8, State: Initialized, Role: FOLLOWER
I20260812 16:32:34.490504 26841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879084ca35cb428594d1d7804540d0c8" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42385 } }
I20260812 16:32:34.490556 26841 raft_consensus.cc:388] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:32:34.490576 26841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:32:34.490603 26841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:34.491017 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 26761
I20260812 16:32:34.491107 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:32:34.491227 26841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879084ca35cb428594d1d7804540d0c8" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42385 } }
I20260812 16:32:34.491305 20409 external_mini_cluster.cc:1428] Setting key 5a0a83349f0ae3a3b553783b58e7e141
I20260812 16:32:34.491351 26841 leader_election.cc:302] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 879084ca35cb428594d1d7804540d0c8; no voters:
I20260812 16:32:34.491590 26841 leader_election.cc:288] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:32:34.491660 26845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:34.491940 26841 sys_catalog.cc:564] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:34.492192 26845 raft_consensus.cc:686] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8 [term 1 LEADER]: Becoming Leader. State: Replica: 879084ca35cb428594d1d7804540d0c8, State: Running, Role: LEADER
I20260812 16:32:34.492347 26845 consensus_queue.cc:237] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879084ca35cb428594d1d7804540d0c8" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42385 } }
I20260812 16:32:34.492794 26850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "879084ca35cb428594d1d7804540d0c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879084ca35cb428594d1d7804540d0c8" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42385 } } }
I20260812 16:32:34.492792 26846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 879084ca35cb428594d1d7804540d0c8. Latest consensus state: current_term: 1 leader_uuid: "879084ca35cb428594d1d7804540d0c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879084ca35cb428594d1d7804540d0c8" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42385 } } }
I20260812 16:32:34.492902 26850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:34.492928 26846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:34.493362 26858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:32:34.493952 26858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:32:34.495678 26858 catalog_manager.cc:1347] Generated new cluster ID: 04bc5f1c2bc346cba1dbf7e6b7ba6a96
I20260812 16:32:34.495735 26858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:32:34 dist-test-slave-mvvj krb5kdc[26745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:34.502486 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:34.492040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43926 (local address 127.19.238.126:42385)
0812 16:32:34.492404 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:34.492414 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:34.492454 (+ 40us) server_negotiation.cc:407] Connection header received
0812 16:32:34.492600 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:34.492609 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:34.492678 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:34.493682 (+ 1004us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:34.493806 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:34.494965 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:34.495990 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:34.496436 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:34.496914 (+ 478us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:34.499078 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:34.499082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:34.499083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:34.499096 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:34.500856 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:34.501256 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:34.501265 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:34.501266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:34.501361 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:34.501596 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:34.501597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:34.501597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:34.501853 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:34.502045 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:34.502116 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 16:32:34.502264 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":91,"threads_started":1}
I20260812 16:32:34.513566 26858 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:32:34.514317 26858 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:34.517746 26858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 879084ca35cb428594d1d7804540d0c8: Generated new TSK 0
I20260812 16:32:34.517938 26858 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
I20260812 16:32:34.554597 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42385
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:37437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:34.640853 26873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:34.641110 26873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:34.641156 26873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:34.642462 26873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:34.642536 26873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:34.642586 26873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:32:34.644260 26873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:37437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:34.644541 26873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:34.644786 26873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:34.650290 26883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:34.650245 26880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:34.650245 26881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:34.650583 26873 server_base.cc:1034] running on GCE node
I20260812 16:32:34.650815 26873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:34.651084 26873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:34.652248 26873 hybrid_clock.cc:648] HybridClock initialized: now 1786552354652240 us; error 36 us; skew 500 ppm
Aug 12 16:32:34 dist-test-slave-mvvj krb5kdc[26745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552354, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:34.655021 26873 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:32:34.655548 26873 webserver.cc:460] Webserver started at http://127.19.238.65:35265/ using document root <none> and password file <none>
I20260812 16:32:34.655820 26873 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:34.655982 26873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:34.656129 26873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:34.657231 26873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "427f420232eb4dd094c7b98592ce64c5"
format_stamp: "Formatted at 2026-08-12 16:32:34 on dist-test-slave-mvvj"
server_key: "41df0e1167e72da4348178e6f4979408"
server_key_iv: "a733d4984c20f711db2b82320ff9f92b"
server_key_version: "encryptionkey@0"
I20260812 16:32:34.657636 26873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "427f420232eb4dd094c7b98592ce64c5"
format_stamp: "Formatted at 2026-08-12 16:32:34 on dist-test-slave-mvvj"
server_key: "41df0e1167e72da4348178e6f4979408"
server_key_iv: "a733d4984c20f711db2b82320ff9f92b"
server_key_version: "encryptionkey@0"
I20260812 16:32:34.659044 26873 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:32:34.659789 26892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:34.660013 26873 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:34.660121 26873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "427f420232eb4dd094c7b98592ce64c5"
format_stamp: "Formatted at 2026-08-12 16:32:34 on dist-test-slave-mvvj"
server_key: "41df0e1167e72da4348178e6f4979408"
server_key_iv: "a733d4984c20f711db2b82320ff9f92b"
server_key_version: "encryptionkey@0"
I20260812 16:32:34.660209 26873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:34.685381 26873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:34.688086 26873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:34.688277 26873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:34.688557 26873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:34.688905 26873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:34.688970 26873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:34.689018 26873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:34.689052 26873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:34.695811 26873 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:40199
I20260812 16:32:34.695915 27018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:40199 every 8 connection(s)
I20260812 16:32:34.696290 26873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:32:34.701225 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 26873
I20260812 16:32:34.701375 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:32:34.701645 20409 external_mini_cluster.cc:1428] Setting key 6bf5243b4dcd078e1eab52ccdebdbe22
Aug 12 16:32:34 dist-test-slave-mvvj krb5kdc[26745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552354, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:34.707590 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:34.697128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:37845 (local address 127.19.238.126:42385)
0812 16:32:34.697201 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:34.697203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:34.697482 (+ 279us) server_negotiation.cc:407] Connection header received
0812 16:32:34.698402 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:34.698404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:34.698441 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:34.698510 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:34.699279 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:34.699795 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:34.700897 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:34.701105 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:34.701188 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:34.704201 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:34.704203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:34.704203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:34.704214 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:34.706012 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:34.706446 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:34.706448 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:34.706448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:34.706515 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:34.706770 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:34.706771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:34.706772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:34.706946 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:34.707027 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:34.707369 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 16:32:34.707479 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 16:32:34.707681 27021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:34.697090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:42385 (local address 127.19.238.65:37845)
0812 16:32:34.697287 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:34.697332 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:34.698284 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:34.698597 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:34.698601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:34.698877 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:34.699089 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:34.699093 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:34.699965 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:34.699966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:34.700790 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:34.700792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:34.701227 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:34.701228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:34.701408 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:34.702115 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:34.702125 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:34.704039 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:34.706108 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:34.706116 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:34.706118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:34.706365 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:34.706587 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:34.706588 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:34.706588 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:34.706697 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:34.707019 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:34.707024 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:34.707201 (+ 177us) client_negotiation.cc:769] Sending connection context
0812 16:32:34.707315 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 16:32:34.707456 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":114,"threads_started":1}
I20260812 16:32:34.708353 27019 heartbeater.cc:347] Connected to a master server at 127.19.238.126:42385
I20260812 16:32:34.708449 27019 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:34.708675 27019 heartbeater.cc:511] Master 127.19.238.126:42385 requested a full tablet report, sending...
I20260812 16:32:34.709362 26798 ts_manager.cc:194] Registered new tserver with Master: 427f420232eb4dd094c7b98592ce64c5 (127.19.238.65:40199)
I20260812 16:32:34.710352 26798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:37845
WARNING: no policy specified for kudu/127.19.238.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
I20260812 16:32:34.731755 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42385
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:37437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:34.824292 27024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:34.824532 27024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:34.824560 27024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:34.825733 27024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:34.825773 27024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:34.825800 27024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:32:34.827318 27024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:37437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:34.827544 27024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:34.827749 27024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:34.833189 27030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:34.833202 27034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:34.833190 27029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:34.833599 27024 server_base.cc:1034] running on GCE node
I20260812 16:32:34.833827 27024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:34.834074 27024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:34.835250 27024 hybrid_clock.cc:648] HybridClock initialized: now 1786552354835235 us; error 29 us; skew 500 ppm
Aug 12 16:32:34 dist-test-slave-mvvj krb5kdc[26745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552354, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:34.838055 27024 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:32:34.838559 27024 webserver.cc:460] Webserver started at http://127.19.238.66:34159/ using document root <none> and password file <none>
I20260812 16:32:34.838774 27024 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:34.838816 27024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:34.838920 27024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:34.840042 27024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "b42c8f2f583e47868e8f45719590acd7"
format_stamp: "Formatted at 2026-08-12 16:32:34 on dist-test-slave-mvvj"
server_key: "d26672044df98ca53a260776b60fee37"
server_key_iv: "1572aab5b674dad28be6b54658f1e392"
server_key_version: "encryptionkey@0"
I20260812 16:32:34.840361 27024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "b42c8f2f583e47868e8f45719590acd7"
format_stamp: "Formatted at 2026-08-12 16:32:34 on dist-test-slave-mvvj"
server_key: "d26672044df98ca53a260776b60fee37"
server_key_iv: "1572aab5b674dad28be6b54658f1e392"
server_key_version: "encryptionkey@0"
I20260812 16:32:34.841784 27024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:32:34.842689 27041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:34.842878 27024 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:34.842978 27024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "b42c8f2f583e47868e8f45719590acd7"
format_stamp: "Formatted at 2026-08-12 16:32:34 on dist-test-slave-mvvj"
server_key: "d26672044df98ca53a260776b60fee37"
server_key_iv: "1572aab5b674dad28be6b54658f1e392"
server_key_version: "encryptionkey@0"
I20260812 16:32:34.843048 27024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:34.875895 27024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:34.878573 27024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:34.878697 27024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:34.878901 27024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:34.879257 27024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:34.879295 27024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:34.879328 27024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:34.879379 27024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:34.885435 27024 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:42217
I20260812 16:32:34.885540 27176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:42217 every 8 connection(s)
I20260812 16:32:34.885879 27024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:32:34.888382 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 27024
I20260812 16:32:34.888506 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:32:34.888703 20409 external_mini_cluster.cc:1428] Setting key f84c582e67d3a68f100c2d5c9c25c41d
Aug 12 16:32:34 dist-test-slave-mvvj krb5kdc[26745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552354, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:34.897558 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:34.886891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:42765 (local address 127.19.238.126:42385)
0812 16:32:34.887027 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:34.887029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:34.887223 (+ 194us) server_negotiation.cc:407] Connection header received
0812 16:32:34.888515 (+ 1292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:34.888517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:34.888532 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:34.888608 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:34.889486 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:34.889998 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:34.891067 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:34.891331 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:34.891396 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:34.894040 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:34.894042 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:34.894042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:34.894051 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:34.896088 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:34.896494 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:34.896495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:34.896496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:34.896565 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:34.896812 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:34.896813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:34.896813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:34.896976 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:34.897054 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:34.897333 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 16:32:34.897435 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260812 16:32:34.897593 27180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:34.886824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:42385 (local address 127.19.238.66:42765)
0812 16:32:34.887065 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:34.887127 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:34.888369 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:34.888691 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:34.888695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:34.889112 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:34.889387 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:34.889393 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:34.890104 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:34.890105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:34.890929 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:34.890931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:34.891419 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:34.891420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:34.891594 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:34.892227 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:34.892241 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:34.893942 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:34.896173 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:34.896180 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:34.896182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:34.896412 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:34.896639 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:34.896640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:34.896640 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:34.896743 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:34.897063 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:34.897069 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:34.897194 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 16:32:34.897286 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 16:32:34.897412 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":98,"threads_started":1}
I20260812 16:32:34.898478 27178 heartbeater.cc:347] Connected to a master server at 127.19.238.126:42385
I20260812 16:32:34.898618 27178 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:34.898828 27178 heartbeater.cc:511] Master 127.19.238.126:42385 requested a full tablet report, sending...
I20260812 16:32:34.899264 26798 ts_manager.cc:194] Registered new tserver with Master: b42c8f2f583e47868e8f45719590acd7 (127.19.238.66:42217)
WARNING: no policy specified for kudu/127.19.238.67@KRBTEST.COM; defaulting to no policy
I20260812 16:32:34.899690 26798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:42765
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
I20260812 16:32:34.919876 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42385
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:37437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:35.003502 27183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:35.003739 27183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:35.003779 27183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:35.004983 27183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:35.005056 27183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:35.005101 27183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:32:35.006604 27183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:37437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:35.006881 27183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:35.007129 27183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:35.012281 27189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:35.012274 27188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:35.012496 27183 server_base.cc:1034] running on GCE node
W20260812 16:32:35.012584 27191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:35.012807 27183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:35.013049 27183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:35.014187 27183 hybrid_clock.cc:648] HybridClock initialized: now 1786552355014168 us; error 30 us; skew 500 ppm
Aug 12 16:32:35 dist-test-slave-mvvj krb5kdc[26745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552355, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:35.016848 27183 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:32:35.017329 27183 webserver.cc:460] Webserver started at http://127.19.238.67:46535/ using document root <none> and password file <none>
I20260812 16:32:35.017606 27183 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:35.017678 27183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:35.017876 27183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:35.019017 27183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "34bf4a21f00a487fa3229fd7b69342f0"
format_stamp: "Formatted at 2026-08-12 16:32:35 on dist-test-slave-mvvj"
server_key: "47c35018ba1217c7109bbbd26ce4381e"
server_key_iv: "b240a22606232130c3f91dd7153ef4c9"
server_key_version: "encryptionkey@0"
I20260812 16:32:35.019368 27183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "34bf4a21f00a487fa3229fd7b69342f0"
format_stamp: "Formatted at 2026-08-12 16:32:35 on dist-test-slave-mvvj"
server_key: "47c35018ba1217c7109bbbd26ce4381e"
server_key_iv: "b240a22606232130c3f91dd7153ef4c9"
server_key_version: "encryptionkey@0"
I20260812 16:32:35.020790 27183 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:32:35.021572 27200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:35.021754 27183 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:35.021842 27183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "34bf4a21f00a487fa3229fd7b69342f0"
format_stamp: "Formatted at 2026-08-12 16:32:35 on dist-test-slave-mvvj"
server_key: "47c35018ba1217c7109bbbd26ce4381e"
server_key_iv: "b240a22606232130c3f91dd7153ef4c9"
server_key_version: "encryptionkey@0"
I20260812 16:32:35.021899 27183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:35.037202 27183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:35.039652 27183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:35.039801 27183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:35.040094 27183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:35.040369 27183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:35.040400 27183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:35.040424 27183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:35.040436 27183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:35.046492 27183 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:36381
I20260812 16:32:35.046561 27338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:36381 every 8 connection(s)
I20260812 16:32:35.046881 27183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
Aug 12 16:32:35 dist-test-slave-mvvj krb5kdc[26745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552355, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:35.055794 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 27183
I20260812 16:32:35.055934 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:32:35.056121 20409 external_mini_cluster.cc:1428] Setting key 6de97a3290383ded3ab191f846ce1234
I20260812 16:32:35.056733 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:35.047725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:59783 (local address 127.19.238.126:42385)
0812 16:32:35.047792 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:35.047794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:35.048046 (+ 252us) server_negotiation.cc:407] Connection header received
0812 16:32:35.049057 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:35.049059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:35.049072 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:35.049122 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:35.049762 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:35.050267 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:35.051287 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:35.051498 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:35.051590 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:35.053862 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:35.053864 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:35.053864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:35.053872 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:35.055181 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:35.055615 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:35.055617 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:35.055617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:35.055680 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:35.055994 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:35.055995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:35.055995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:35.056126 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:35.056207 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:35.056533 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 16:32:35.056641 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:32:35.056898 27341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:35.047690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:42385 (local address 127.19.238.67:59783)
0812 16:32:35.047927 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:35.047963 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:35.048941 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:35.049197 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:35.049200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:35.049459 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:35.049657 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:35.049662 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:35.050368 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:35.050369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:35.051192 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:35.051194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:35.051603 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:35.051604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:35.051733 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:35.052321 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:35.052329 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:35.053760 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:35.055267 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:35.055275 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:35.055277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:35.055519 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:35.055747 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:35.055748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:35.055748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:35.055917 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:35.056195 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:35.056202 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:35.056369 (+ 167us) client_negotiation.cc:769] Sending connection context
0812 16:32:35.056490 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 16:32:35.056659 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":153,"threads_started":1}
I20260812 16:32:35.057404 27339 heartbeater.cc:347] Connected to a master server at 127.19.238.126:42385
I20260812 16:32:35.057516 27339 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:35.057704 27339 heartbeater.cc:511] Master 127.19.238.126:42385 requested a full tablet report, sending...
I20260812 16:32:35.058053 26798 ts_manager.cc:194] Registered new tserver with Master: 34bf4a21f00a487fa3229fd7b69342f0 (127.19.238.67:36381)
I20260812 16:32:35.058413 26798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:59783
I20260812 16:32:35.068521 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/test-admin.keytab.
Aug 12 16:32:35 dist-test-slave-mvvj krb5kdc[26745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:35.086777 20409 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 16:32:35.088675 27346 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:35.087732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47074 (local address 127.19.238.65:40199)
0812 16:32:35.087912 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:35.087917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:35.087931 (+ 14us) server_negotiation.cc:407] Connection header received
0812 16:32:35.087967 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:35.087969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:35.087983 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:35.088068 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:35.088560 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47074: BlockingRecv error: recv got EOF from 127.0.0.1:47074 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":102,"threads_started":1}
W20260812 16:32:35.088724 20409 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/krb5cc' of type FILE
Aug 12 16:32:35 dist-test-slave-mvvj krb5kdc[26745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:35.089874 20409 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 16:32:35 dist-test-slave-mvvj krb5kdc[26745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:35.090801 20409 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 16:32:35 dist-test-slave-mvvj krb5kdc[26745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.65@KRBTEST.COM
I20260812 16:32:35.108706 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:35.091361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47078 (local address 127.19.238.65:40199)
0812 16:32:35.091425 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:35.091427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:35.091466 (+ 39us) server_negotiation.cc:407] Connection header received
0812 16:32:35.091562 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:35.091563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:35.091576 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:35.091652 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:35.092210 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:35.093049 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:35.093916 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:35.094263 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:35.094359 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:35.105559 (+ 11200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:35.105580 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:35.105580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:35.105591 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:35.107187 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:35.107646 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:35.107650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:35.107650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:35.107710 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:35.108019 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:35.108020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:35.108021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:35.108193 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:35.108382 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:35.108510 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 16:32:35.108604 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260812 16:32:35.111254 27346 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:35.110412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47084 (local address 127.19.238.65:40199)
0812 16:32:35.110699 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:35.110702 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:35.110710 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:32:35.110733 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:35.110734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:35.110748 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:35.110813 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:35.111199 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47084: BlockingRecv error: recv got EOF from 127.0.0.1:47084 (error 108)
Metrics: {"server-negotiator.queue_time_us":265}
W20260812 16:32:35.111390 20409 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/krb5cc' of type FILE
Aug 12 16:32:35 dist-test-slave-mvvj krb5kdc[26745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:35.112726 20409 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 16:32:35 dist-test-slave-mvvj krb5kdc[26745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:35.113734 20409 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 16:32:35 dist-test-slave-mvvj krb5kdc[26745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.65@KRBTEST.COM
I20260812 16:32:35.120834 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:35.114300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47088 (local address 127.19.238.65:40199)
0812 16:32:35.114340 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:35.114342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:35.114354 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:32:35.114426 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:35.114426 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:35.114434 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:35.114472 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:35.114819 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:35.115313 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:35.116074 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:35.116273 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:35.116324 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:35.118399 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:35.118400 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:35.118400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:35.118407 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:35.119700 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:35.120135 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:35.120136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:35.120137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:35.120193 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:35.120376 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:35.120376 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:35.120377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:35.120473 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:35.120568 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:35.120643 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 16:32:35.120710 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 16:32:35.122421 27346 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:35.121905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47102 (local address 127.19.238.65:40199)
0812 16:32:35.121943 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:35.121945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:35.121999 (+ 54us) server_negotiation.cc:407] Connection header received
0812 16:32:35.122064 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:35.122064 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:35.122072 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:35.122115 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:35.122380 (+ 265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47102: BlockingRecv error: recv got EOF from 127.0.0.1:47102 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 16:32:35.122591 20409 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786552327177731-20409-0/krb5kdc/krb5cc' of type FILE
Aug 12 16:32:35 dist-test-slave-mvvj krb5kdc[26745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:35.123679 20409 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 16:32:35 dist-test-slave-mvvj krb5kdc[26745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:35.124706 20409 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 16:32:35 dist-test-slave-mvvj krb5kdc[26745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.65@KRBTEST.COM
I20260812 16:32:35.131942 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:35.125282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47108 (local address 127.19.238.65:40199)
0812 16:32:35.125341 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:35.125343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:35.125352 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:32:35.125409 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:35.125410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:35.125417 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:35.125492 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:35.125871 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:35.126352 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:35.127114 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:35.127313 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:35.127384 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:35.129496 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:35.129497 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:35.129498 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:35.129505 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:35.130876 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:35.131184 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:35.131185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:35.131185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:35.131235 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:35.131404 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:35.131405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:35.131405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:35.131494 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:35.131564 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:35.131716 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 16:32:35.131797 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:32:35.132740 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 26873
I20260812 16:32:35.141785 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 27024
I20260812 16:32:35.148367 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 27183
I20260812 16:32:35.154793 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 26761
2026-08-12T16:32:35Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2057 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:32:35 dist-test-slave-mvvj krb5kdc[27359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:32:35 dist-test-slave-mvvj krb5kdc[27359](info): set up 2 sockets
Aug 12 16:32:35 dist-test-slave-mvvj krb5kdc[27359](info): commencing operation
krb5kdc: starting...
W20260812 16:32:36.287945 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:32:36 dist-test-slave-mvvj krb5kdc[27359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:32:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:32:36Z Disabled control of system clock
WARNING: no policy specified for oryx/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.126.keytab.
Entry for principal oryx/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.126.keytab.
I20260812 16:32:36.420058 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:46705
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.126.keytab
--principal=oryx/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:42945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:46705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:36.512116 27372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:36.512380 27372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:36.512422 27372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:36.513684 27372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:36.513759 27372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:36.513788 27372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:36.513824 27372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:36.513852 27372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:32:36.515619 27372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:42945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:46705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.126.keytab
--principal=oryx/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:46705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:36.515962 27372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:36.516220 27372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:36.521678 27381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:36.521678 27378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:36.521780 27377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:36.522011 27372 server_base.cc:1034] running on GCE node
I20260812 16:32:36.522269 27372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:36.522574 27372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:36.523743 27372 hybrid_clock.cc:648] HybridClock initialized: now 1786552356523739 us; error 41 us; skew 500 ppm
Aug 12 16:32:36 dist-test-slave-mvvj krb5kdc[27359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552356, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:36.526414 27372 init.cc:377] Logged in from keytab as oryx/127.19.238.126@KRBTEST.COM (short username oryx)
I20260812 16:32:36.526928 27372 webserver.cc:460] Webserver started at http://127.19.238.126:37013/ using document root <none> and password file <none>
I20260812 16:32:36.527182 27372 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:36.527248 27372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:36.527387 27372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:36.528543 27372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f"
format_stamp: "Formatted at 2026-08-12 16:32:36 on dist-test-slave-mvvj"
server_key: "a7e5216e91179eeda6f565c0f6d8b015"
server_key_iv: "e4a2363d24d795a3763f3bdb51313103"
server_key_version: "encryptionkey@0"
I20260812 16:32:36.528911 27372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f"
format_stamp: "Formatted at 2026-08-12 16:32:36 on dist-test-slave-mvvj"
server_key: "a7e5216e91179eeda6f565c0f6d8b015"
server_key_iv: "e4a2363d24d795a3763f3bdb51313103"
server_key_version: "encryptionkey@0"
I20260812 16:32:36.530318 27372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:32:36.531121 27389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:36.531318 27372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:36.531423 27372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f"
format_stamp: "Formatted at 2026-08-12 16:32:36 on dist-test-slave-mvvj"
server_key: "a7e5216e91179eeda6f565c0f6d8b015"
server_key_iv: "e4a2363d24d795a3763f3bdb51313103"
server_key_version: "encryptionkey@0"
I20260812 16:32:36.531527 27372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:36.552237 27372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:36.554733 27372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:36.554920 27372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:36.559147 27372 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:46705
I20260812 16:32:36.559190 27450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:46705 every 8 connection(s)
I20260812 16:32:36.559592 27372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:32:36.560729 27451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:36.563406 27451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f: Bootstrap starting.
I20260812 16:32:36.564141 27451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:36.564462 27451 log.cc:824] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:36.565231 27451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f: No bootstrap required, opened a new log
I20260812 16:32:36.566336 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 27372
I20260812 16:32:36.566480 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:32:36.566687 20409 external_mini_cluster.cc:1428] Setting key 8dcf0b44bb3db4c78cdf4feadcf29a3f
I20260812 16:32:36.568209 27451 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } }
I20260812 16:32:36.568406 27451 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:36.568526 27451 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b8e1b2644347ccbb22cfc7f9622c8f, State: Initialized, Role: FOLLOWER
I20260812 16:32:36.568687 27451 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } }
I20260812 16:32:36.568786 27451 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:32:36.568843 27451 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:32:36.568898 27451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:36.569687 27451 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } }
I20260812 16:32:36.569808 27451 leader_election.cc:302] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9b8e1b2644347ccbb22cfc7f9622c8f; no voters:
I20260812 16:32:36.569983 27451 leader_election.cc:288] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:32:36.570011 27456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:36.570186 27456 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [term 1 LEADER]: Becoming Leader. State: Replica: a9b8e1b2644347ccbb22cfc7f9622c8f, State: Running, Role: LEADER
I20260812 16:32:36.570251 27451 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:36.570307 27456 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } }
I20260812 16:32:36.570932 27457 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } }
I20260812 16:32:36.571029 27457 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:36.571326 27458 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9b8e1b2644347ccbb22cfc7f9622c8f. Latest consensus state: current_term: 1 leader_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } }
I20260812 16:32:36.571401 27458 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:36.571678 27466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:32:36.572295 27466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:32:36.574579 27466 catalog_manager.cc:1347] Generated new cluster ID: e2c224ef7aa047f6b3d0c62416405557
I20260812 16:32:36.574643 27466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:32:36 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM
I20260812 16:32:36.579802 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:36.567270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40986 (local address 127.19.238.126:46705)
0812 16:32:36.567521 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:36.567531 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:36.567572 (+ 41us) server_negotiation.cc:407] Connection header received
0812 16:32:36.568168 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:36.568178 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:36.568248 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:36.568427 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:36.569257 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:36.570851 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:36.572186 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:36.572774 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:36.572892 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:36.575848 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:36.575854 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:36.575854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:36.575866 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:36.577945 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:36.578422 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:36.578425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:36.578425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:36.578555 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:36.578791 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:36.578792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:36.578792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:36.579101 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:36.579258 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:36.579453 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 16:32:36.579593 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":106,"threads_started":1}
I20260812 16:32:36.585359 27466 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:32:36.585950 27466 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:36.588225 27466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f: Generated new TSK 0
I20260812 16:32:36.588419 27466 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.65.keytab.
Entry for principal oryx/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.65.keytab.
I20260812 16:32:36.619405 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:46705
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.65.keytab
--principal=oryx/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:42945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:36.710429 27481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:36.710668 27481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:36.710695 27481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:36.711968 27481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:36.712013 27481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:36.712042 27481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:32:36.713601 27481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:42945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.65.keytab
--principal=oryx/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:46705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:36.713856 27481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:36.714139 27481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:36.719491 27487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:36.719539 27491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:36.719542 27488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:36.719798 27481 server_base.cc:1034] running on GCE node
I20260812 16:32:36.720057 27481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:36.720322 27481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:36.721508 27481 hybrid_clock.cc:648] HybridClock initialized: now 1786552356721493 us; error 45 us; skew 500 ppm
Aug 12 16:32:36 dist-test-slave-mvvj krb5kdc[27359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552356, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:36.724249 27481 init.cc:377] Logged in from keytab as oryx/127.19.238.65@KRBTEST.COM (short username oryx)
I20260812 16:32:36.724838 27481 webserver.cc:460] Webserver started at http://127.19.238.65:46087/ using document root <none> and password file <none>
I20260812 16:32:36.725204 27481 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:36.725286 27481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:36.725454 27481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:36.726653 27481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "c8c477cc39284473822fb1229d47a24e"
format_stamp: "Formatted at 2026-08-12 16:32:36 on dist-test-slave-mvvj"
server_key: "3f70348c0de693f8a5a813063222c9e9"
server_key_iv: "28f61cf082e93eb9f8490d2e3403b9c3"
server_key_version: "encryptionkey@0"
I20260812 16:32:36.727003 27481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "c8c477cc39284473822fb1229d47a24e"
format_stamp: "Formatted at 2026-08-12 16:32:36 on dist-test-slave-mvvj"
server_key: "3f70348c0de693f8a5a813063222c9e9"
server_key_iv: "28f61cf082e93eb9f8490d2e3403b9c3"
server_key_version: "encryptionkey@0"
I20260812 16:32:36.728500 27481 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:32:36.729218 27499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:36.729403 27481 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:36.729481 27481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "c8c477cc39284473822fb1229d47a24e"
format_stamp: "Formatted at 2026-08-12 16:32:36 on dist-test-slave-mvvj"
server_key: "3f70348c0de693f8a5a813063222c9e9"
server_key_iv: "28f61cf082e93eb9f8490d2e3403b9c3"
server_key_version: "encryptionkey@0"
I20260812 16:32:36.729575 27481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:36.750784 27481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:36.753084 27481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:36.753234 27481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:36.753526 27481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:36.753865 27481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:36.753897 27481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:36.753921 27481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:36.753980 27481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:36.760274 27481 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:43035
I20260812 16:32:36.760334 27634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:43035 every 8 connection(s)
I20260812 16:32:36.760699 27481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:32:36.765939 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 27481
I20260812 16:32:36.766084 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:32:36.766287 20409 external_mini_cluster.cc:1428] Setting key 155a1ea627ccb9d28f82392c1808e3c3
Aug 12 16:32:36 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552356, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM
I20260812 16:32:36.771230 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:36.761500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:35775 (local address 127.19.238.126:46705)
0812 16:32:36.761568 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:36.761570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:36.761798 (+ 228us) server_negotiation.cc:407] Connection header received
0812 16:32:36.762301 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:36.762302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:36.762315 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:36.762372 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:36.763433 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:36.763993 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:36.764995 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:36.765206 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:36.765267 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:36.768066 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:36.768068 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:36.768069 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:36.768077 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:36.769542 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:36.770065 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:36.770066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:36.770067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:36.770131 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:36.770442 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:36.770443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:36.770443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:36.770613 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:36.770688 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:36.770964 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 16:32:36.771086 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:32:36.771449 27638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:36.761472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.65:35775)
0812 16:32:36.761661 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:36.761707 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:36.762219 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:36.762439 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:36.762443 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:36.763036 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:36.763236 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:36.763241 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:36.764074 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:36.764075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:36.764904 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:36.764906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:36.765284 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:36.765285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:36.765385 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:36.766009 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:36.766017 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:36.767976 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:36.769631 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:36.769640 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:36.769642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:36.769982 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:36.770201 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:36.770202 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:36.770202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:36.770327 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:36.770692 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:36.770697 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:36.770818 (+ 121us) client_negotiation.cc:769] Sending connection context
0812 16:32:36.770926 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 16:32:36.771131 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":108,"threads_started":1}
I20260812 16:32:36.771973 27635 heartbeater.cc:347] Connected to a master server at 127.19.238.126:46705
I20260812 16:32:36.772080 27635 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:36.772264 27635 heartbeater.cc:511] Master 127.19.238.126:46705 requested a full tablet report, sending...
I20260812 16:32:36.772816 27408 ts_manager.cc:194] Registered new tserver with Master: c8c477cc39284473822fb1229d47a24e (127.19.238.65:43035)
I20260812 16:32:36.773502 27408 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:35775
WARNING: no policy specified for oryx/127.19.238.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.66.keytab.
Entry for principal oryx/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.66.keytab.
I20260812 16:32:36.795185 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:46705
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.66.keytab
--principal=oryx/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:42945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:36.894367 27641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:36.894596 27641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:36.894621 27641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:36.895802 27641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:36.895915 27641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:36.895972 27641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:32:36.897480 27641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:42945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.66.keytab
--principal=oryx/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:46705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:36.897718 27641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:36.897975 27641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:36.903146 27646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:36.903245 27648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:36.903146 27651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:36.903501 27641 server_base.cc:1034] running on GCE node
I20260812 16:32:36.903699 27641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:36.903975 27641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:36.905133 27641 hybrid_clock.cc:648] HybridClock initialized: now 1786552356905117 us; error 28 us; skew 500 ppm
Aug 12 16:32:36 dist-test-slave-mvvj krb5kdc[27359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552356, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:36.907769 27641 init.cc:377] Logged in from keytab as oryx/127.19.238.66@KRBTEST.COM (short username oryx)
I20260812 16:32:36.908270 27641 webserver.cc:460] Webserver started at http://127.19.238.66:44269/ using document root <none> and password file <none>
I20260812 16:32:36.908516 27641 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:36.908565 27641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:36.908739 27641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:36.909845 27641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "bec516f368e74cef842a68aad5c20bda"
format_stamp: "Formatted at 2026-08-12 16:32:36 on dist-test-slave-mvvj"
server_key: "e315600c9b409b9b642beb896665eac4"
server_key_iv: "fe8db5a08edfdba8240de2840d349b68"
server_key_version: "encryptionkey@0"
I20260812 16:32:36.910218 27641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "bec516f368e74cef842a68aad5c20bda"
format_stamp: "Formatted at 2026-08-12 16:32:36 on dist-test-slave-mvvj"
server_key: "e315600c9b409b9b642beb896665eac4"
server_key_iv: "fe8db5a08edfdba8240de2840d349b68"
server_key_version: "encryptionkey@0"
I20260812 16:32:36.911698 27641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:32:36.912563 27658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:36.912761 27641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:36.912882 27641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "bec516f368e74cef842a68aad5c20bda"
format_stamp: "Formatted at 2026-08-12 16:32:36 on dist-test-slave-mvvj"
server_key: "e315600c9b409b9b642beb896665eac4"
server_key_iv: "fe8db5a08edfdba8240de2840d349b68"
server_key_version: "encryptionkey@0"
I20260812 16:32:36.912955 27641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:36.930051 27641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:36.932154 27641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:36.932319 27641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:36.932591 27641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:36.932925 27641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:36.932958 27641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:36.933012 27641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:36.933043 27641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:36.939113 27641 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:36503
I20260812 16:32:36.939162 27789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:36503 every 8 connection(s)
I20260812 16:32:36.939492 27641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:32:36.941247 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 27641
I20260812 16:32:36.941433 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:32:36.941650 20409 external_mini_cluster.cc:1428] Setting key c93f4a26b16ab1b14e01c1a34c4fc0ee
Aug 12 16:32:36 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552356, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM
I20260812 16:32:36.949944 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:36.940264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:59195 (local address 127.19.238.126:46705)
0812 16:32:36.940349 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:36.940352 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:36.940525 (+ 173us) server_negotiation.cc:407] Connection header received
0812 16:32:36.941019 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:36.941021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:36.941049 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:36.941129 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:36.942313 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:36.943060 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:36.944102 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:36.944412 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:36.944496 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:36.946946 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:36.946948 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:36.946948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:36.946957 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:36.948486 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:36.948939 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:36.948940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:36.948941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:36.949002 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:36.949261 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:36.949262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:36.949262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:36.949387 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:36.949465 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:36.949751 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 16:32:36.949841 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 16:32:36.950141 27792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:36.940210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.66:59195)
0812 16:32:36.940400 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:36.940437 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:36.940931 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:36.941181 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:36.941184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:36.941946 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:36.942214 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:36.942220 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:36.943158 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:36.943159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:36.944024 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:36.944026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:36.944515 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:36.944517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:36.944656 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:36.945222 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:36.945230 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:36.946843 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:36.948588 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:36.948596 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:36.948598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:36.948834 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:36.949080 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:36.949081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:36.949081 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:36.949190 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:36.949464 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:36.949472 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:36.949612 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 16:32:36.949717 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 16:32:36.949882 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":106,"threads_started":1}
I20260812 16:32:36.950640 27790 heartbeater.cc:347] Connected to a master server at 127.19.238.126:46705
I20260812 16:32:36.950738 27790 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:36.950944 27790 heartbeater.cc:511] Master 127.19.238.126:46705 requested a full tablet report, sending...
I20260812 16:32:36.951329 27408 ts_manager.cc:194] Registered new tserver with Master: bec516f368e74cef842a68aad5c20bda (127.19.238.66:36503)
WARNING: no policy specified for oryx/127.19.238.67@KRBTEST.COM; defaulting to no policy
I20260812 16:32:36.951723 27408 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:59195
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.238.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.67.keytab.
Entry for principal oryx/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.67.keytab.
I20260812 16:32:36.971552 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:46705
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.67.keytab
--principal=oryx/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:42945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:37.055961 27795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:37.056164 27795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:37.056187 27795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:37.057381 27795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:37.057426 27795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:37.057452 27795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:32:37.058848 27795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:42945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.67.keytab
--principal=oryx/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:46705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:37.059044 27795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:37.059226 27795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:37.064932 27801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:37.064952 27802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:37.065122 27795 server_base.cc:1034] running on GCE node
W20260812 16:32:37.065181 27804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:37.065412 27795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:37.065624 27795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:37.066790 27795 hybrid_clock.cc:648] HybridClock initialized: now 1786552357066789 us; error 43 us; skew 500 ppm
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552357, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:37.069684 27795 init.cc:377] Logged in from keytab as oryx/127.19.238.67@KRBTEST.COM (short username oryx)
I20260812 16:32:37.070343 27795 webserver.cc:460] Webserver started at http://127.19.238.67:41229/ using document root <none> and password file <none>
I20260812 16:32:37.070627 27795 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:37.070708 27795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:37.070858 27795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:37.072357 27795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "a60ad8f19246497b85c74a6f7d5d2695"
format_stamp: "Formatted at 2026-08-12 16:32:37 on dist-test-slave-mvvj"
server_key: "a3d5e7ff1af25eb264c4a6f59d75c6fc"
server_key_iv: "5c5e7a0b3d2d628981610b7d599b94af"
server_key_version: "encryptionkey@0"
I20260812 16:32:37.072780 27795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "a60ad8f19246497b85c74a6f7d5d2695"
format_stamp: "Formatted at 2026-08-12 16:32:37 on dist-test-slave-mvvj"
server_key: "a3d5e7ff1af25eb264c4a6f59d75c6fc"
server_key_iv: "5c5e7a0b3d2d628981610b7d599b94af"
server_key_version: "encryptionkey@0"
I20260812 16:32:37.074247 27795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:32:37.075111 27812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:37.075295 27795 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:37.075455 27795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "a60ad8f19246497b85c74a6f7d5d2695"
format_stamp: "Formatted at 2026-08-12 16:32:37 on dist-test-slave-mvvj"
server_key: "a3d5e7ff1af25eb264c4a6f59d75c6fc"
server_key_iv: "5c5e7a0b3d2d628981610b7d599b94af"
server_key_version: "encryptionkey@0"
I20260812 16:32:37.075546 27795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:37.104076 27795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:37.106168 27795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:37.106355 27795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:37.106632 27795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:37.106999 27795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:37.107069 27795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:37.107118 27795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:37.107156 27795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:37.113699 27795 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:42721
I20260812 16:32:37.113745 27944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:42721 every 8 connection(s)
I20260812 16:32:37.114140 27795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:32:37.117515 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 27795
I20260812 16:32:37.117606 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:32:37.117800 20409 external_mini_cluster.cc:1428] Setting key 89ffcdd530d874984eee8cdfb75fecd6
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552357, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM
I20260812 16:32:37.125721 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.114997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:40423 (local address 127.19.238.126:46705)
0812 16:32:37.115075 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.115078 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.115248 (+ 170us) server_negotiation.cc:407] Connection header received
0812 16:32:37.115938 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.115940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.115952 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.116010 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.117181 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.118072 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.119621 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.120014 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.120089 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.122567 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.122569 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.122569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.122579 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.124102 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.124611 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.124612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.124613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.124682 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.124976 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.124977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.124977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.125108 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.125183 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.125509 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.125618 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 16:32:37.125933 27949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.114946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.67:40423)
0812 16:32:37.115116 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.115160 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.115759 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.116084 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.116088 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.116768 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.117054 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.117062 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.118209 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.118211 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.119519 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.119522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.120172 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.120174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.120337 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.120834 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.120844 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.122394 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.124201 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.124209 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.124212 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.124521 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.124768 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.124769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.124769 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.124892 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.125189 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.125196 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.125351 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.125484 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.125668 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
I20260812 16:32:37.126479 27947 heartbeater.cc:347] Connected to a master server at 127.19.238.126:46705
I20260812 16:32:37.126578 27947 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:37.126789 27947 heartbeater.cc:511] Master 127.19.238.126:46705 requested a full tablet report, sending...
I20260812 16:32:37.127254 27408 ts_manager.cc:194] Registered new tserver with Master: a60ad8f19246497b85c74a6f7d5d2695 (127.19.238.67:42721)
I20260812 16:32:37.127681 27408 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:40423
I20260812 16:32:37.130043 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.19.238.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.238.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.238.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.125.keytab.
Entry for principal oryx/127.19.238.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.125.keytab.
I20260812 16:32:37.160130 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.125:34857
--webserver_interface=127.19.238.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.125.keytab
--principal=oryx/127.19.238.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:42945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:46705,127.19.238.125:34857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:37.245988 27952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:37.246248 27952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:37.246294 27952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:37.247525 27952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:37.247597 27952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:37.247622 27952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:37.247654 27952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:37.247680 27952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:32:37.249292 27952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:42945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:46705,127.19.238.125:34857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.125.keytab
--principal=oryx/127.19.238.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.125:34857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.238.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
W20260812 16:32:37.249446 27952 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.19.238.126:46705,127.19.238.125:34857'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 16:32:37.249626 27952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:37.249879 27952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:37.254897 27960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:37.255020 27963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:37.255057 27961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:37.255614 27952 server_base.cc:1034] running on GCE node
I20260812 16:32:37.255923 27952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:37.256249 27952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:37.257422 27952 hybrid_clock.cc:648] HybridClock initialized: now 1786552357257416 us; error 37 us; skew 500 ppm
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552357, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:37.260279 27952 init.cc:377] Logged in from keytab as oryx/127.19.238.125@KRBTEST.COM (short username oryx)
I20260812 16:32:37.260924 27952 webserver.cc:460] Webserver started at http://127.19.238.125:36281/ using document root <none> and password file <none>
I20260812 16:32:37.261258 27952 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:37.261336 27952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:37.261511 27952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:37.262629 27952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/data/instance:
uuid: "e4f1d932a8304008a7162555424cafbe"
format_stamp: "Formatted at 2026-08-12 16:32:37 on dist-test-slave-mvvj"
server_key: "7308fe60fed182a020d5ceaeb7e20094"
server_key_iv: "4bc1949e59094703bd4cce4d9c3b418e"
server_key_version: "encryptionkey@0"
I20260812 16:32:37.263017 27952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/wal/instance:
uuid: "e4f1d932a8304008a7162555424cafbe"
format_stamp: "Formatted at 2026-08-12 16:32:37 on dist-test-slave-mvvj"
server_key: "7308fe60fed182a020d5ceaeb7e20094"
server_key_iv: "4bc1949e59094703bd4cce4d9c3b418e"
server_key_version: "encryptionkey@0"
I20260812 16:32:37.264608 27952 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:32:37.265508 27970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:37.265709 27952 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:37.265808 27952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/wal
uuid: "e4f1d932a8304008a7162555424cafbe"
format_stamp: "Formatted at 2026-08-12 16:32:37 on dist-test-slave-mvvj"
server_key: "7308fe60fed182a020d5ceaeb7e20094"
server_key_iv: "4bc1949e59094703bd4cce4d9c3b418e"
server_key_version: "encryptionkey@0"
I20260812 16:32:37.265869 27952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:37.285441 27952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:37.287822 27952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:37.288050 27952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:37.292272 27952 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.125:34857
I20260812 16:32:37.292304 28034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.125:34857 every 8 connection(s)
I20260812 16:32:37.292706 27952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/data/info.pb
I20260812 16:32:37.293742 28035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:37.295369 28035 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:32:37.296042 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 27952
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552357, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.125@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.238.125@KRBTEST.COM
I20260812 16:32:37.305941 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.296217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40994 (local address 127.19.238.126:46705)
0812 16:32:37.296307 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.296310 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.296550 (+ 240us) server_negotiation.cc:407] Connection header received
0812 16:32:37.297303 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.297305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.297319 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.297386 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.297988 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.298501 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.299567 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.299919 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.299982 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.302546 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.302547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.302548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.302556 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.304317 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.304749 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.304751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.304751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.304817 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.305118 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.305118 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.305119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.305298 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.305376 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.305735 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.305824 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 16:32:37.306357 28037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.296190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:40994)
0812 16:32:37.296392 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.296443 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.297172 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.297425 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.297427 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.297697 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.297860 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.297864 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.298608 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.298609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.299471 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.299473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.300001 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.300002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.300166 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.300778 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.300786 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.302405 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.304418 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.304424 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.304426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.304671 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.304892 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.304893 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.304893 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.305038 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.305385 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.305390 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.305576 (+ 186us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.305703 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.305833 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":4,"thread_start_us":102,"threads_started":1}
I20260812 16:32:37.306515 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.296738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40066 (local address 127.19.238.125:34857)
0812 16:32:37.296884 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.296890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.296924 (+ 34us) server_negotiation.cc:407] Connection header received
0812 16:32:37.297174 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.297183 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.297213 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.297307 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.297942 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.298949 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.299770 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.300268 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.300338 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.303082 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.303089 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.303090 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.303096 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.304962 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.305330 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.305333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.305334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.305398 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.305648 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.305649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.305649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.305841 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.306003 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.306328 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.306409 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":121,"thread_start_us":72,"threads_started":1}
I20260812 16:32:37.307164 28035 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552357, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.125@KRBTEST.COM for oryx/127.19.238.125@KRBTEST.COM
I20260812 16:32:37.313293 28037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.307411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:34857 (local address 127.0.0.1:40078)
0812 16:32:37.307465 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.307476 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.307512 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.307614 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.307615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.307728 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.307760 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.307761 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.308417 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.308418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.308881 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.308883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.309215 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.309215 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.309256 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.309617 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.309621 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.310782 (+ 1161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.312414 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.312415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.312416 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.312620 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.312796 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.312797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.312798 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.312850 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.313048 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.313049 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.313083 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.313134 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.313210 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 16:32:37.313342 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.307461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40078 (local address 127.19.238.125:34857)
0812 16:32:37.307496 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.307497 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.307506 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:32:37.307560 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.307560 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.307568 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.307612 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.307816 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.308332 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.308949 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.309153 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.309205 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.310862 (+ 1657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.310863 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.310864 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.310869 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.312319 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.312690 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.312691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.312692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.312739 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.312900 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.312900 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.312901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.312997 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.313044 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.313149 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.313254 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":16}
I20260812 16:32:37.314883 28035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: Bootstrap starting.
I20260812 16:32:37.315761 28035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:37.316205 28035 log.cc:824] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:37.317009 28035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: No bootstrap required, opened a new log
I20260812 16:32:37.318511 28035 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } }
I20260812 16:32:37.318667 28035 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:37.318712 28035 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f1d932a8304008a7162555424cafbe, State: Initialized, Role: FOLLOWER
I20260812 16:32:37.318903 28035 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } }
I20260812 16:32:37.319229 28041 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } } }
I20260812 16:32:37.319306 28035 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:37.319320 28041 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [sys.catalog]: This master's current role is: FOLLOWER
W20260812 16:32:37.321118 28055 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:32:37.321184 28055 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:32:37.323460 27952 master_runner.cc:428] Detected that this master e4f1d932a8304008a7162555424cafbe is joining an existing cluster
I20260812 16:32:37.323535 27952 master_runner.cc:432] Initiating AddMaster RPC to add 127.19.238.125:34857
I20260812 16:32:37.332790 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.325098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:34857 (local address 127.0.0.1:40094)
0812 16:32:37.325249 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.325259 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.325317 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.325462 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.325462 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.325703 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.325758 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.325760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.326714 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.326716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.327623 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.327626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.328183 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.328184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.328251 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.328616 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.328621 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.329281 (+ 660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.331395 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.331396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.331396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.331671 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.331925 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.331925 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.331926 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.331988 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.332306 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.332308 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.332362 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.332481 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.332638 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":30,"thread_start_us":86,"threads_started":1}
I20260812 16:32:37.333125 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.324791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41004 (local address 127.19.238.126:46705)
0812 16:32:37.324863 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.324866 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.324979 (+ 113us) server_negotiation.cc:407] Connection header received
0812 16:32:37.325040 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.325041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.325054 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.325119 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.325709 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.326633 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.327505 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.327808 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.327965 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.329518 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.329519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.329520 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.329528 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.331386 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.331923 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.331924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.331924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.331986 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.332507 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.332508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.332508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.332647 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.332695 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.332943 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.333044 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:32:37.333333 28063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.324731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:41004)
0812 16:32:37.324894 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.324908 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.324978 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.325180 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.325182 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.325478 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.325581 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.325583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.326730 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.326731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.327404 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.327406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.327975 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.327977 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.328056 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.328882 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.328887 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.329337 (+ 450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.331505 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.331506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.331507 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.331790 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.332339 (+ 549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.332339 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.332340 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.332413 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.332743 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.332744 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.332782 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.333007 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.333115 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":107,"threads_started":1}
I20260812 16:32:37.332798 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.325092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40094 (local address 127.19.238.125:34857)
0812 16:32:37.325133 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.325134 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.325329 (+ 195us) server_negotiation.cc:407] Connection header received
0812 16:32:37.325358 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.325359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.325375 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.325441 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.325842 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.326618 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.327710 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.328088 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.328168 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.329396 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.329398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.329398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.329405 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.331289 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.331754 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.331755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.331755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.331814 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.332061 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.332062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.332063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.332213 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.332312 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.332508 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.332700 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":24}
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552356, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.126@KRBTEST.COM for oryx/127.19.238.125@KRBTEST.COM
I20260812 16:32:37.344308 28067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.336233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:34857 (local address 127.0.0.1:40096)
0812 16:32:37.336393 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.336458 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.336538 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.336688 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.336691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.337055 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.337140 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.337143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.337765 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.337766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.338750 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.338753 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.339149 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.339150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.339247 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.339727 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.339733 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.341303 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.343038 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.343046 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.343048 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.343281 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.343547 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.343548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.343549 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.343632 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.343968 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.343976 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.344029 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.344118 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.344230 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260812 16:32:37.344340 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.336282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40096 (local address 127.19.238.125:34857)
0812 16:32:37.336326 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.336329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.336548 (+ 219us) server_negotiation.cc:407] Connection header received
0812 16:32:37.336619 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.336620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.336629 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.336673 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.337216 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.337678 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.338852 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.339063 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.339129 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.341391 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.341392 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.341393 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.341401 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.342932 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.343373 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.343374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.343375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.343440 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.343712 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.343712 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.343713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.343892 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.343963 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.344163 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.344245 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:32:37.345333 27408 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.19.238.125:34857: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e4f1d932a8304008a7162555424cafbe"
member_type: NON_VOTER
last_known_addr {
host: "127.19.238.125"
port: 34857
}
attrs {
promote: true
}
}
dest_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f"
cas_config_opid_index: -1
I20260812 16:32:37.345785 27408 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: NON_VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: true } }
I20260812 16:32:37.346350 28069 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER e4f1d932a8304008a7162555424cafbe (127.19.238.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: NON_VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: true } } }
I20260812 16:32:37.346760 28006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:37.347414 28069 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.19.238.125:34857
I20260812 16:32:37.347507 28069 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: NON_VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: true } } }
I20260812 16:32:37.347585 28069 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:37.347728 28006 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 1 FOLLOWER]: Refusing update from remote peer a9b8e1b2644347ccbb22cfc7f9622c8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 16:32:37.348004 27952 master.cc:537] Master@127.19.238.125:34857 shutting down...
I20260812 16:32:37.348191 28068 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: NON_VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 16:32:37.349011 27394 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 16:32:37.349524 27394 consensus_peers.cc:487] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f -> Peer e4f1d932a8304008a7162555424cafbe (127.19.238.125:34857): Couldn't send request to peer e4f1d932a8304008a7162555424cafbe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 16:32:37.351014 27952 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:37.351140 27952 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:37.351217 27952 tablet_replica.cc:330] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: stopping tablet replica
I20260812 16:32:37.364495 27952 master.cc:552] Master@127.19.238.125:34857 shutdown complete.
I20260812 16:32:37.364586 27952 master_runner.cc:305] Clearing existing system tablet
I20260812 16:32:37.364745 27952 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:37.364814 27952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:37.365556 27952 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260812 16:32:37.366130 28073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:37.366276 27952 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:37.366413 27952 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:32:37.366466 27952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/wal
uuid: "e4f1d932a8304008a7162555424cafbe"
format_stamp: "Formatted at 2026-08-12 16:32:37 on dist-test-slave-mvvj"
server_key: "7308fe60fed182a020d5ceaeb7e20094"
server_key_iv: "4bc1949e59094703bd4cce4d9c3b418e"
server_key_version: "encryptionkey@0"
I20260812 16:32:37.367201 27952 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:37.368976 27952 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 16:32:37.369050 27952 log.cc:1196] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 16:32:37.369297 27952 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: Deleting consensus metadata
I20260812 16:32:37.369524 27952 master_runner.cc:315] Copying system tablet from 127.19.238.126:46705
I20260812 16:32:37.370213 27952 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: tablet copy: Beginning tablet copy session from remote peer at address 127.19.238.126:46705
I20260812 16:32:37.377556 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.370750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:41018)
0812 16:32:37.370886 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.370899 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.370998 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.371172 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.371173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.371358 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.371423 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.371424 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.372449 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.372451 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.373078 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.373081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.373583 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.373585 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.373649 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.374198 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.374203 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.374515 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.376456 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.376457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.376457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.376660 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.376879 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.376880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.376880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.377000 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.377284 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.377284 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.377328 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.377381 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.377470 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
I20260812 16:32:37.377602 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.370852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41018 (local address 127.19.238.126:46705)
0812 16:32:37.370912 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.370915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.370981 (+ 66us) server_negotiation.cc:407] Connection header received
0812 16:32:37.371084 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.371085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.371097 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.371165 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.371530 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.372313 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.373171 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.373489 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.373565 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.374607 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.374609 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.374609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.374616 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.376348 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.376740 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.376741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.376742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.376810 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.377077 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.377078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.377078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.377217 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.377290 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.377417 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.377506 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 16:32:37.378080 27432 tablet_copy_service.cc:140] P a9b8e1b2644347ccbb22cfc7f9622c8f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e4f1d932a8304008a7162555424cafbe ({username='oryx', principal='oryx/127.19.238.125@KRBTEST.COM'} at 127.0.0.1:41018)
I20260812 16:32:37.378197 27432 tablet_copy_service.cc:161] P a9b8e1b2644347ccbb22cfc7f9622c8f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e4f1d932a8304008a7162555424cafbe at {username='oryx', principal='oryx/127.19.238.125@KRBTEST.COM'} at 127.0.0.1:41018: session id = e4f1d932a8304008a7162555424cafbe-00000000000000000000000000000000
I20260812 16:32:37.378957 27432 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 16:32:37.379618 27952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 16:32:37.380736 27952 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: tablet copy: Starting download of 0 data blocks...
I20260812 16:32:37.380897 27952 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: tablet copy: Starting download of 1 WAL segments...
I20260812 16:32:37.381832 27952 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 16:32:37.382714 27432 tablet_copy_service.cc:342] P a9b8e1b2644347ccbb22cfc7f9622c8f: Request end of tablet copy session e4f1d932a8304008a7162555424cafbe-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.19.238.125@KRBTEST.COM'} at 127.0.0.1:41018
I20260812 16:32:37.382778 27432 tablet_copy_service.cc:433] P a9b8e1b2644347ccbb22cfc7f9622c8f: ending tablet copy session e4f1d932a8304008a7162555424cafbe-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e4f1d932a8304008a7162555424cafbe
I20260812 16:32:37.385844 27952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:37.387077 28082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:37.387149 28085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:37.387310 27952 server_base.cc:1034] running on GCE node
W20260812 16:32:37.387306 28083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:37.387601 27952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:37.387789 27952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:37.388939 27952 hybrid_clock.cc:648] HybridClock initialized: now 1786552357388939 us; error 36 us; skew 500 ppm
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552357, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:37.390424 27952 init.cc:377] Logged in from keytab as oryx/127.19.238.125@KRBTEST.COM (short username oryx)
I20260812 16:32:37.390754 27952 webserver.cc:460] Webserver started at http://127.19.238.125:37199/ using document root <none> and password file <none>
I20260812 16:32:37.390899 27952 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:37.390939 27952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:37.391626 27952 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260812 16:32:37.392282 28097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:37.392406 27952 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:37.392455 27952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/wal
uuid: "e4f1d932a8304008a7162555424cafbe"
format_stamp: "Formatted at 2026-08-12 16:32:37 on dist-test-slave-mvvj"
server_key: "7308fe60fed182a020d5ceaeb7e20094"
server_key_iv: "4bc1949e59094703bd4cce4d9c3b418e"
server_key_version: "encryptionkey@0"
I20260812 16:32:37.392524 27952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:37.402127 27952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:37.403766 27952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:37.407126 27952 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.125:34857
I20260812 16:32:37.407419 28153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.125:34857 every 8 connection(s)
I20260812 16:32:37.407547 27952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-1/data/info.pb
I20260812 16:32:37.408624 28154 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:32:37.409068 28154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: Bootstrap starting.
I20260812 16:32:37.411643 28154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:32:37.411906 28154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: Bootstrap complete.
I20260812 16:32:37.412200 28154 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: NON_VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: true } }
I20260812 16:32:37.412307 28154 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e4f1d932a8304008a7162555424cafbe, State: Initialized, Role: LEARNER
I20260812 16:32:37.412348 28154 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: NON_VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: true } }
I20260812 16:32:37.412544 28154 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:37.413204 28159 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: NON_VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: true } } }
I20260812 16:32:37.413331 28159 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [sys.catalog]: This master's current role is: LEARNER
I20260812 16:32:37.414503 28171 catalog_manager.cc:1259] Loaded cluster ID: e2c224ef7aa047f6b3d0c62416405557
I20260812 16:32:37.414551 28171 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: loading cluster ID for follower catalog manager: success
I20260812 16:32:37.415558 28171 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: acquiring CA information for follower catalog manager: success
I20260812 16:32:37.415915 28171 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:32:37.416603 28156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.408208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40122 (local address 127.19.238.125:34857)
0812 16:32:37.408335 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.408337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.408345 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:32:37.408368 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.408369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.408379 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.408445 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.409166 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.410138 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.411157 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.411488 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.411572 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.413145 (+ 1573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.413147 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.413147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.413155 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.415184 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.415678 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.415679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.415680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.415748 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.416031 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.416033 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.416033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.416143 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.416254 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.416390 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.416505 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":82,"threads_started":1}
WARNING: no policy specified for oryx/127.19.238.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.238.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.238.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.124.keytab.
Entry for principal oryx/127.19.238.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.124.keytab.
I20260812 16:32:37.450912 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.124:39739
--webserver_interface=127.19.238.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.124.keytab
--principal=oryx/127.19.238.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:42945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:46705,127.19.238.125:34857,127.19.238.124:39739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:37.549893 28175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:37.550118 28175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:37.550143 28175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:37.551249 28175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:37.551286 28175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:37.551301 28175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:37.551319 28175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:37.551332 28175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:32:37.552799 28175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:42945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:46705,127.19.238.125:34857,127.19.238.124:39739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.124.keytab
--principal=oryx/127.19.238.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.124:39739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.238.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:37.553000 28175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:37.553179 28175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:37.558254 28180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:37.558270 28175 server_base.cc:1034] running on GCE node
W20260812 16:32:37.558274 28183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:37.558254 28181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:37.558765 28175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:37.559083 28175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:37.560250 28175 hybrid_clock.cc:648] HybridClock initialized: now 1786552357560223 us; error 42 us; skew 500 ppm
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552357, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:37.562999 28175 init.cc:377] Logged in from keytab as oryx/127.19.238.124@KRBTEST.COM (short username oryx)
I20260812 16:32:37.563447 28175 webserver.cc:460] Webserver started at http://127.19.238.124:35027/ using document root <none> and password file <none>
I20260812 16:32:37.563686 28175 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:37.563750 28175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:37.563954 28175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:37.565044 28175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/data/instance:
uuid: "18228093a7184cbdb85bb16d8e59efa6"
format_stamp: "Formatted at 2026-08-12 16:32:37 on dist-test-slave-mvvj"
server_key: "fb286b0f4830906773ed72da88a9a5af"
server_key_iv: "514027730fe648d3053bb14dd47b0988"
server_key_version: "encryptionkey@0"
I20260812 16:32:37.565423 28175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/wal/instance:
uuid: "18228093a7184cbdb85bb16d8e59efa6"
format_stamp: "Formatted at 2026-08-12 16:32:37 on dist-test-slave-mvvj"
server_key: "fb286b0f4830906773ed72da88a9a5af"
server_key_iv: "514027730fe648d3053bb14dd47b0988"
server_key_version: "encryptionkey@0"
I20260812 16:32:37.566831 28175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:32:37.567660 28190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:37.567891 28175 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:37.568038 28175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/wal
uuid: "18228093a7184cbdb85bb16d8e59efa6"
format_stamp: "Formatted at 2026-08-12 16:32:37 on dist-test-slave-mvvj"
server_key: "fb286b0f4830906773ed72da88a9a5af"
server_key_iv: "514027730fe648d3053bb14dd47b0988"
server_key_version: "encryptionkey@0"
I20260812 16:32:37.568138 28175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:37.581677 28175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:37.584444 28175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:37.584627 28175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:37.588974 28175 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.124:39739
I20260812 16:32:37.589010 28255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.124:39739 every 8 connection(s)
I20260812 16:32:37.589416 28175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/data/info.pb
I20260812 16:32:37.590569 28256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:37.592304 28256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:32:37.598242 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 28175
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552357, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.124@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM
I20260812 16:32:37.602841 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.593254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41022 (local address 127.19.238.126:46705)
0812 16:32:37.593387 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.593390 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.593524 (+ 134us) server_negotiation.cc:407] Connection header received
0812 16:32:37.594100 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.594102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.594115 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.594183 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.594855 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.595389 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.596469 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.596676 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.596773 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.599459 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.599461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.599462 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.599471 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.601254 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.601704 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.601706 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.601706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.601780 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.602065 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.602066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.602067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.602225 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.602306 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.602693 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.602769 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260812 16:32:37.603093 28258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.593121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:41022)
0812 16:32:37.593362 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.593415 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.593981 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.594258 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.594261 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.594541 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.594736 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.594740 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.595490 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.595491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.596374 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.596377 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.596799 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.596808 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.596934 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.597499 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.597508 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.599326 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.601336 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.601343 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.601345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.601623 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.601855 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.601856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.601856 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.601981 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.602302 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.602308 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.602520 (+ 212us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.602654 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.602821 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":117,"threads_started":1}
I20260812 16:32:37.604213 28256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.238.124@KRBTEST.COM
I20260812 16:32:37.608192 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.598878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34900 (local address 127.19.238.124:39739)
0812 16:32:37.599087 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.599091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.599219 (+ 128us) server_negotiation.cc:407] Connection header received
0812 16:32:37.599248 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.599250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.599275 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.599356 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.600033 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.600704 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.601484 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.601926 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.601981 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.604877 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.604886 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.604886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.604903 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.606733 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.607200 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.607203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.607203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.607262 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.607499 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.607500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.607500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.607697 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.607932 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.607983 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.608070 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":131,"threads_started":1}
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552357, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.124@KRBTEST.COM for oryx/127.19.238.125@KRBTEST.COM
I20260812 16:32:37.611795 28258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.604506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:34857 (local address 127.0.0.1:40126)
0812 16:32:37.604569 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.604582 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.604630 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.604846 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.604847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.605047 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.605113 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.605115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.606124 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.606125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.606991 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.606994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.607501 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.607503 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.607587 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.608025 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.608030 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.609530 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.610987 (+ 1457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.610987 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.610988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.611153 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.611338 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.611339 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.611339 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.611390 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.611583 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.611584 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.611618 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.611667 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.611736 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 16:32:37.611821 28156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.604623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40126 (local address 127.19.238.125:34857)
0812 16:32:37.604702 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.604704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.604714 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:37.604741 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.604742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.604754 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.604818 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.605386 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.606043 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.607084 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.607425 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.607490 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.609648 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.609649 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.609650 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.609657 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.610910 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.611222 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.611223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.611224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.611278 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.611430 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.611431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.611431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.611532 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.611583 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.611685 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.611755 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 16:32:37.612437 28256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552357, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.124@KRBTEST.COM for oryx/127.19.238.124@KRBTEST.COM
I20260812 16:32:37.619225 28258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.612739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:39739 (local address 127.0.0.1:34912)
0812 16:32:37.612794 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.612805 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.612857 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.613022 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.613024 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.613161 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.613189 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.613190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.613967 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.613968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.614488 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.614489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.614930 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.614931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.614969 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.615371 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.615375 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.616621 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.618203 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.618204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.618204 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.618385 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.618589 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.618590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.618590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.618645 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.618905 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.618906 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.618941 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.619020 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.619116 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 16:32:37.619274 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.612813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34912 (local address 127.19.238.124:39739)
0812 16:32:37.612848 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.612850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.612859 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:32:37.612924 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.612924 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.612933 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.612973 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.613261 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.613896 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.614596 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.614848 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.614932 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.616720 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.616721 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.616721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.616730 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.618100 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.618468 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.618469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.618470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.618524 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.618710 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.618711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.618712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.618820 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.618893 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.619041 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.619164 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":18}
I20260812 16:32:37.620784 28256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: Bootstrap starting.
I20260812 16:32:37.621662 28256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:37.621960 28256 log.cc:824] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:37.622696 28256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: No bootstrap required, opened a new log
I20260812 16:32:37.624248 28256 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } }
I20260812 16:32:37.624428 28256 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:37.624475 28256 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18228093a7184cbdb85bb16d8e59efa6, State: Initialized, Role: FOLLOWER
I20260812 16:32:37.624603 28256 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } }
I20260812 16:32:37.624914 28256 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:37.624887 28262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } } }
I20260812 16:32:37.624993 28262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 16:32:37.626595 28273 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:32:37.626657 28273 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:32:37.629231 28175 master_runner.cc:428] Detected that this master 18228093a7184cbdb85bb16d8e59efa6 is joining an existing cluster
I20260812 16:32:37.629277 28175 master_runner.cc:432] Initiating AddMaster RPC to add 127.19.238.124:39739
I20260812 16:32:37.637573 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.630437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34928 (local address 127.19.238.124:39739)
0812 16:32:37.630476 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.630477 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.630683 (+ 206us) server_negotiation.cc:407] Connection header received
0812 16:32:37.630762 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.630763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.630770 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.630796 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.631329 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.632103 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.633277 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.633554 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.633613 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.634842 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.634843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.634844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.634850 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.636416 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.636833 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.636834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.636835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.636898 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.637111 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.637111 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.637112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.637211 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.637260 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.637444 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.637512 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 16:32:37.637995 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.630376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:39739 (local address 127.0.0.1:34928)
0812 16:32:37.630632 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.630641 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.630701 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.630851 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.630851 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.631126 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.631246 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.631249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.632218 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.632219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.633150 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.633153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.633718 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.633719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.633780 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.634231 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.634237 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.634728 (+ 491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.636511 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.636513 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.636514 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.636750 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.636962 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.636963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.636964 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.637036 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.637276 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.637277 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.637343 (+ 66us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.637431 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.637586 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":83,"thread_start_us":85,"threads_started":1}
I20260812 16:32:37.657039 28156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.630294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40130 (local address 127.19.238.125:34857)
0812 16:32:37.630348 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.630350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.630827 (+ 477us) server_negotiation.cc:407] Connection header received
0812 16:32:37.630934 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.630935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.630946 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.630993 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.631401 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.632205 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.633413 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.633710 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.633749 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.653619 (+ 19870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.653622 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.653622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.653634 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.655526 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.656046 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.656047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.656048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.656115 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.656408 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.656409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.656409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.656568 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.656645 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.656836 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.656927 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 16:32:37.657085 28282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.630515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:34857 (local address 127.0.0.1:40130)
0812 16:32:37.630754 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.630764 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.630854 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.631005 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.631006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.631258 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.631311 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.631312 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.632323 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.632323 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.633256 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.633259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.633829 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.633829 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.633894 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.652726 (+ 18832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.652733 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.653453 (+ 720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.655644 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.655646 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.655647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.655965 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.656214 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.656215 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.656215 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.656282 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.656649 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.656650 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.656722 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.656798 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.656940 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":27,"thread_start_us":59,"threads_started":1}
I20260812 16:32:37.657250 28279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.630253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:41036)
0812 16:32:37.630470 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.630481 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.630544 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.630694 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.630695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.631114 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.631258 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.631260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.632297 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.632298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.633259 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.633261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.633794 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.633794 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.633861 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.653141 (+ 19280us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.653146 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.653655 (+ 509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.655776 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.655778 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.655778 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.656086 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.656315 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.656316 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.656316 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.656381 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.656682 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.656683 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.656733 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.656811 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.657001 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":110,"thread_start_us":81,"threads_started":1}
I20260812 16:32:37.657660 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.630299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41036 (local address 127.19.238.126:46705)
0812 16:32:37.630353 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.630356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.630535 (+ 179us) server_negotiation.cc:407] Connection header received
0812 16:32:37.630605 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.630606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.630619 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.630673 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.631437 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.632190 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.633375 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.633671 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.633711 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.653761 (+ 20050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.653764 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.653764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.653780 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.655678 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.656173 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.656175 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.656175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.656243 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.656450 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.656451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.656451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.656590 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.656652 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.657523 (+ 871us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.657598 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:32:37.659682 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 27481
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552356, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.126@KRBTEST.COM for oryx/127.19.238.124@KRBTEST.COM
I20260812 16:32:37.675000 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:43035
--local_ip_for_outbound_sockets=127.19.238.65
--tserver_master_addrs=127.19.238.126:46705
--webserver_port=46087
--webserver_interface=127.19.238.65
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.65.keytab
--principal=oryx/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:42945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.238.126:46705,127.19.238.125:34857
--tserver_master_addrs=127.19.238.126:46705,127.19.238.125:34857,127.19.238.124:39739 with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:32:37.676403 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.666623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34932 (local address 127.19.238.124:39739)
0812 16:32:37.668002 (+ 1379us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.668004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.668018 (+ 14us) server_negotiation.cc:407] Connection header received
0812 16:32:37.668077 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.668078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.668094 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.668181 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.668675 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.669345 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.670416 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.670735 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.670828 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.673335 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.673337 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.673338 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.673347 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.675061 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.675518 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.675519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.675520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.675587 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.675805 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.675806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.675806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.675970 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.676047 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.676222 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.676307 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1348}
I20260812 16:32:37.676416 28067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.666245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:39739 (local address 127.0.0.1:34932)
0812 16:32:37.666346 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.666357 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.666398 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.668272 (+ 1874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.668274 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.668526 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.668584 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.668587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.669448 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.669450 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.670305 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.670310 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.670896 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.670898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.670992 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.671554 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.671561 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.673231 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.675203 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.675204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.675205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.675444 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.675666 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.675667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.675667 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.675735 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.676053 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.676054 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.676106 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.676185 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.676309 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 16:32:37.677439 27408 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.19.238.124:39739: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6"
member_type: NON_VOTER
last_known_addr {
host: "127.19.238.124"
port: 39739
}
attrs {
promote: true
}
}
dest_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f"
cas_config_opid_index: 5
I20260812 16:32:37.677755 27408 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: NON_VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: true } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: NON_VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: true } }
I20260812 16:32:37.678210 28069 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 18228093a7184cbdb85bb16d8e59efa6 (127.19.238.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: NON_VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: true } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: NON_VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: true } } }
I20260812 16:32:37.679432 28069 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.19.238.124:39739
I20260812 16:32:37.679522 28069 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: NON_VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: true } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: NON_VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: true } } }
I20260812 16:32:37.679607 28069 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:37.680069 28175 master.cc:537] Master@127.19.238.124:39739 shutting down...
I20260812 16:32:37.680269 28224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:37.682056 28224 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 1 FOLLOWER]: Refusing update from remote peer a9b8e1b2644347ccbb22cfc7f9622c8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 16:32:37.682510 28068 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: NON_VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 16:32:37.683317 27391 consensus_peers.cc:487] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f -> Peer 18228093a7184cbdb85bb16d8e59efa6 (127.19.238.124:39739): Couldn't send request to peer 18228093a7184cbdb85bb16d8e59efa6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 16:32:37.684222 28175 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:37.684326 28175 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:37.684377 28175 tablet_replica.cc:330] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: stopping tablet replica
I20260812 16:32:37.687469 28175 master.cc:552] Master@127.19.238.124:39739 shutdown complete.
I20260812 16:32:37.687516 28175 master_runner.cc:305] Clearing existing system tablet
I20260812 16:32:37.687654 28175 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:37.687696 28175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:37.689284 28067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.678221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:34857 (local address 127.0.0.1:40144)
0812 16:32:37.683523 (+ 5302us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.683537 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.683596 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.683902 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.683903 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.683905 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:37.683970 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.683973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.685102 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.685103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.687890 (+ 2787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.687895 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.688932 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.688933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.689018 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.689055 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.689108 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.689176 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5279}
I20260812 16:32:37.689589 28156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.678379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40144 (local address 127.19.238.125:34857)
0812 16:32:37.678438 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.678440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.683666 (+ 5226us) server_negotiation.cc:407] Connection header received
0812 16:32:37.683704 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.683705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.683734 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.683797 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:37.684064 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.684985 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.688005 (+ 3020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.688714 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.688800 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.688886 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.689460 (+ 574us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.689520 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 16:32:37.689833 28126 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 1 LEARNER]: Refusing update from remote peer a9b8e1b2644347ccbb22cfc7f9622c8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 16:32:37.690045 28068 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: NON_VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 16:32:37.690634 28126 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 18228093a7184cbdb85bb16d8e59efa6 (127.19.238.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: NON_VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: true } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: NON_VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: true } } }
I20260812 16:32:37.692055 28159 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9b8e1b2644347ccbb22cfc7f9622c8f. Latest consensus state: current_term: 1 leader_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: NON_VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: true } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: NON_VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: true } } }
I20260812 16:32:37.692152 28159 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [sys.catalog]: This master's current role is: LEARNER
I20260812 16:32:37.692189 28159 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: NON_VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: true } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: NON_VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: true } } }
I20260812 16:32:37.692260 28159 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [sys.catalog]: This master's current role is: LEARNER
I20260812 16:32:37.692557 28068 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f: attempting to promote NON_VOTER e4f1d932a8304008a7162555424cafbe to VOTER
I20260812 16:32:37.692782 28068 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: NON_VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: true } }
I20260812 16:32:37.693588 28126 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 1 LEARNER]: Refusing update from remote peer a9b8e1b2644347ccbb22cfc7f9622c8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 16:32:37.693719 28288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.693411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:39739 (local address 127.0.0.1:34944)
0812 16:32:37.693554 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.693616 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:39739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260812 16:32:37.693781 28069 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 16:32:37.694267 28288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.694104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:39739 (local address 127.0.0.1:34954)
0812 16:32:37.694214 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.694239 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:39739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
W20260812 16:32:37.694401 27391 consensus_peers.cc:487] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f -> Peer 18228093a7184cbdb85bb16d8e59efa6 (127.19.238.124:39739): Couldn't send request to peer 18228093a7184cbdb85bb16d8e59efa6. Status: Network error: Client connection negotiation failed: client connection to 127.19.238.124:39739: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 16:32:37.694355 28069 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e4f1d932a8304008a7162555424cafbe (127.19.238.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: NON_VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: true } } }
I20260812 16:32:37.695241 28069 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: NON_VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: true } } }
I20260812 16:32:37.695320 28069 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:37.695291 28287 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: NON_VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: true } } }
I20260812 16:32:37.695595 28287 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:37.695569 28126 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e4f1d932a8304008a7162555424cafbe (127.19.238.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: NON_VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: true } } }
I20260812 16:32:37.696077 28175 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.000s sys 0.000s
I20260812 16:32:37.696532 28159 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: NON_VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: true } } }
I20260812 16:32:37.696616 28159 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:32:37.696975 28290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:37.697105 28175 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:37.697165 28175 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:32:37.697194 28175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/wal
uuid: "18228093a7184cbdb85bb16d8e59efa6"
format_stamp: "Formatted at 2026-08-12 16:32:37 on dist-test-slave-mvvj"
server_key: "fb286b0f4830906773ed72da88a9a5af"
server_key_iv: "514027730fe648d3053bb14dd47b0988"
server_key_version: "encryptionkey@0"
I20260812 16:32:37.697995 28175 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:37.699343 28175 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 16:32:37.699429 28175 log.cc:1196] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 16:32:37.699627 28175 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: Deleting consensus metadata
I20260812 16:32:37.699898 28175 master_runner.cc:315] Copying system tablet from 127.19.238.126:46705
I20260812 16:32:37.700551 28175 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: tablet copy: Beginning tablet copy session from remote peer at address 127.19.238.126:46705
I20260812 16:32:37.708315 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.700935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41046 (local address 127.19.238.126:46705)
0812 16:32:37.700977 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.700980 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.701059 (+ 79us) server_negotiation.cc:407] Connection header received
0812 16:32:37.701128 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.701128 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.701139 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.701199 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.701557 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.702322 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.703568 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.703924 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.704031 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.705217 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.705219 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.705219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.705227 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.706940 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.707368 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.707369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.707370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.707438 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.707673 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.707674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.707674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.707817 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.707912 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.708131 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.708221 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:32:37.708689 28297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.700869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:41046)
0812 16:32:37.700999 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.701009 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.701075 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.701213 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.701214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.701417 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.701481 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.701483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.702585 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.702586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.703456 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.703460 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.704033 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.704035 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.704116 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.704652 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.704658 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.705115 (+ 457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.707052 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.707054 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.707054 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.707284 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.707517 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.707519 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.707519 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.707600 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.707957 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.707959 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.708012 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.708471 (+ 459us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.708597 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260812 16:32:37.709015 27432 tablet_copy_service.cc:140] P a9b8e1b2644347ccbb22cfc7f9622c8f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 18228093a7184cbdb85bb16d8e59efa6 ({username='oryx', principal='oryx/127.19.238.124@KRBTEST.COM'} at 127.0.0.1:41046)
I20260812 16:32:37.709106 27432 tablet_copy_service.cc:161] P a9b8e1b2644347ccbb22cfc7f9622c8f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 18228093a7184cbdb85bb16d8e59efa6 at {username='oryx', principal='oryx/127.19.238.124@KRBTEST.COM'} at 127.0.0.1:41046: session id = 18228093a7184cbdb85bb16d8e59efa6-00000000000000000000000000000000
I20260812 16:32:37.709758 27432 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 16:32:37.710289 28175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 16:32:37.711705 28175 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: tablet copy: Starting download of 0 data blocks...
I20260812 16:32:37.711953 28175 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: tablet copy: Starting download of 1 WAL segments...
I20260812 16:32:37.712885 28175 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 16:32:37.713956 27432 tablet_copy_service.cc:342] P a9b8e1b2644347ccbb22cfc7f9622c8f: Request end of tablet copy session 18228093a7184cbdb85bb16d8e59efa6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.19.238.124@KRBTEST.COM'} at 127.0.0.1:41046
I20260812 16:32:37.714020 27432 tablet_copy_service.cc:433] P a9b8e1b2644347ccbb22cfc7f9622c8f: ending tablet copy session 18228093a7184cbdb85bb16d8e59efa6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 18228093a7184cbdb85bb16d8e59efa6
I20260812 16:32:37.716928 28175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:37.718178 28298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:37.721058 28299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:37.721020 28301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:37.721529 28175 server_base.cc:1034] running on GCE node
I20260812 16:32:37.721655 28175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:37.721812 28175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:37.722960 28175 hybrid_clock.cc:648] HybridClock initialized: now 1786552357722938 us; error 40 us; skew 500 ppm
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552357, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:37.724573 28175 init.cc:377] Logged in from keytab as oryx/127.19.238.124@KRBTEST.COM (short username oryx)
I20260812 16:32:37.724853 28175 webserver.cc:460] Webserver started at http://127.19.238.124:39273/ using document root <none> and password file <none>
I20260812 16:32:37.724984 28175 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:37.725023 28175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:37.725771 28175 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:37.726428 28309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:37.726552 28175 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:37.726598 28175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/wal
uuid: "18228093a7184cbdb85bb16d8e59efa6"
format_stamp: "Formatted at 2026-08-12 16:32:37 on dist-test-slave-mvvj"
server_key: "fb286b0f4830906773ed72da88a9a5af"
server_key_iv: "514027730fe648d3053bb14dd47b0988"
server_key_version: "encryptionkey@0"
I20260812 16:32:37.726666 28175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:37.742509 28175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:37.744472 28175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:37.747733 28175 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.124:39739
I20260812 16:32:37.747979 28368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.124:39739 every 8 connection(s)
I20260812 16:32:37.748172 28175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/master-2/data/info.pb
I20260812 16:32:37.749101 28369 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:32:37.749589 28369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: Bootstrap starting.
I20260812 16:32:37.752565 28369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:32:37.752784 28369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: Bootstrap complete.
I20260812 16:32:37.752995 28369 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: NON_VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: true } }
I20260812 16:32:37.753093 28369 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 18228093a7184cbdb85bb16d8e59efa6, State: Initialized, Role: LEARNER
I20260812 16:32:37.753129 28369 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: NON_VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: true } }
I20260812 16:32:37.753310 28369 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:37.753330 28373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: NON_VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: true } } }
I20260812 16:32:37.753531 28373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [sys.catalog]: This master's current role is: LEARNER
I20260812 16:32:37.754977 28388 catalog_manager.cc:1259] Loaded cluster ID: e2c224ef7aa047f6b3d0c62416405557
I20260812 16:32:37.755076 28388 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: loading cluster ID for follower catalog manager: success
I20260812 16:32:37.756286 28388 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: acquiring CA information for follower catalog manager: success
I20260812 16:32:37.756608 28388 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 16:32:37.782586 28283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:37.782826 28283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:37.782868 28283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:37.784111 28283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:37.784173 28283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:37.784219 28283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:32:37.785849 28283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:42945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.65.keytab
--principal=oryx/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:43035
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=46087
--tserver_master_addrs=127.19.238.126:46705,127.19.238.125:34857,127.19.238.124:39739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:37.786132 28283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:37.786394 28283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:37.792642 28394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:37.792611 28396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:37.792647 28393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:37.793134 28283 server_base.cc:1034] running on GCE node
I20260812 16:32:37.793365 28283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:37.793666 28283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:37.794848 28283 hybrid_clock.cc:648] HybridClock initialized: now 1786552357794834 us; error 33 us; skew 500 ppm
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552357, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:37.798084 28283 init.cc:377] Logged in from keytab as oryx/127.19.238.65@KRBTEST.COM (short username oryx)
I20260812 16:32:37.798638 28283 webserver.cc:460] Webserver started at http://127.19.238.65:46087/ using document root <none> and password file <none>
I20260812 16:32:37.798909 28283 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:37.799007 28283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:37.800484 28283 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:32:37.801177 28404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:37.801378 28283 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:37.801492 28283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "c8c477cc39284473822fb1229d47a24e"
format_stamp: "Formatted at 2026-08-12 16:32:36 on dist-test-slave-mvvj"
server_key: "3f70348c0de693f8a5a813063222c9e9"
server_key_iv: "28f61cf082e93eb9f8490d2e3403b9c3"
server_key_version: "encryptionkey@0"
I20260812 16:32:37.801883 28283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:37.821022 28283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:37.823191 28283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:37.823374 28283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:37.823627 28283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:37.824074 28283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:37.824115 28283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:37.824139 28283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:37.824151 28283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:37.831012 28283 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:43035
I20260812 16:32:37.831095 28537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:43035 every 8 connection(s)
I20260812 16:32:37.831471 28283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:32:37.832818 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 28283
I20260812 16:32:37.833014 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 27641
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552357, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552357, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.125@KRBTEST.COM
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552357, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.124@KRBTEST.COM
I20260812 16:32:37.847751 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.832628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:58103 (local address 127.19.238.126:46705)
0812 16:32:37.832685 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.832687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.832803 (+ 116us) server_negotiation.cc:407] Connection header received
0812 16:32:37.834657 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.834659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.834675 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.834710 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.838035 (+ 3325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.838785 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.839969 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.840239 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.840275 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.843893 (+ 3618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.843895 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.843895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.843904 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.845961 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.846534 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.846535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.846536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.846601 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.846927 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.846928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.846929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.847082 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.847138 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.847535 (+ 397us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.847627 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 16:32:37.848397 28544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.832263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.65:58103)
0812 16:32:37.832663 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.832714 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.834519 (+ 1805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.834868 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.834873 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.835333 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.835718 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.835725 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.838880 (+ 3155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.838881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.839876 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.839879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.840346 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.840346 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.840746 (+ 400us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.841121 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.841125 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.843770 (+ 2645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.846061 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.846072 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.846075 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.846429 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.846675 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.846676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.846677 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.846828 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.847174 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.847182 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.847368 (+ 186us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.847942 (+ 574us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.848137 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":1,"thread_start_us":105,"threads_started":1}
I20260812 16:32:37.848407 28156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.832593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:54291 (local address 127.19.238.125:34857)
0812 16:32:37.832965 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.832968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.832977 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:32:37.836449 (+ 3472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.836451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.836467 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.836507 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.837062 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.838056 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.839486 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.839747 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.839804 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.844190 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.844192 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.844192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.844200 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.846762 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.847224 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.847226 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.847226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.847285 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.847505 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.847506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.847507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.847652 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.847695 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.848224 (+ 529us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.848309 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343}
I20260812 16:32:37.849035 28540 heartbeater.cc:347] Connected to a master server at 127.19.238.126:46705
I20260812 16:32:37.849179 28540 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:37.849303 28547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.832262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:34857 (local address 127.19.238.65:54291)
0812 16:32:37.832673 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.832714 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.836282 (+ 3568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.836601 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.836603 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.836846 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.836932 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.836935 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.838173 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.838174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.839388 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.839391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.839928 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.839929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.840126 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.840933 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.840947 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.843592 (+ 2645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.846846 (+ 3254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.846847 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.846848 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.847051 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.847358 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.847358 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.847359 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.847430 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.847762 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.847763 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.848085 (+ 322us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.848906 (+ 821us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.849026 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":3,"thread_start_us":123,"threads_started":1}
I20260812 16:32:37.849421 28540 heartbeater.cc:511] Master 127.19.238.126:46705 requested a full tablet report, sending...
I20260812 16:32:37.849956 28542 heartbeater.cc:347] Connected to a master server at 127.19.238.125:34857
I20260812 16:32:37.849992 27408 ts_manager.cc:194] Re-registered known tserver with Master: c8c477cc39284473822fb1229d47a24e (127.19.238.65:43035)
I20260812 16:32:37.850085 28542 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:37.850199 28542 heartbeater.cc:511] Master 127.19.238.125:34857 requested a full tablet report, sending...
I20260812 16:32:37.850468 27408 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:58103
I20260812 16:32:37.850603 28115 ts_manager.cc:194] Registered new tserver with Master: c8c477cc39284473822fb1229d47a24e (127.19.238.65:43035)
I20260812 16:32:37.850924 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.832263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:39739 (local address 127.19.238.65:52375)
0812 16:32:37.832744 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.832755 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.834529 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.834870 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.834873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.835992 (+ 1119us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.836071 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.836073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.837193 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.837193 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.838443 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.838446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.838994 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.838994 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.843267 (+ 4273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.845767 (+ 2500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.845773 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.847442 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.849724 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.849726 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.849726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.849958 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.850187 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.850187 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.850188 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.850249 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.850564 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.850565 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.850608 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.850673 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.850785 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":4389,"thread_start_us":60,"threads_started":1}
I20260812 16:32:37.850946 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:36503
--local_ip_for_outbound_sockets=127.19.238.66
--tserver_master_addrs=127.19.238.126:46705
--webserver_port=44269
--webserver_interface=127.19.238.66
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.66.keytab
--principal=oryx/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:42945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.238.126:46705,127.19.238.125:34857
--tserver_master_addrs=127.19.238.126:46705,127.19.238.125:34857,127.19.238.124:39739 with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:32:37.851284 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.832453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:52375 (local address 127.19.238.124:39739)
0812 16:32:37.832579 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.832581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.832901 (+ 320us) server_negotiation.cc:407] Connection header received
0812 16:32:37.834699 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.834700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.834714 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.834782 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.836167 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.837073 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.838540 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.838866 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.838926 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.847573 (+ 8647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.847575 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.847576 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.847584 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.849625 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.850050 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.850051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.850052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.850118 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.850321 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.850321 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.850322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.850467 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.850537 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.851053 (+ 516us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.851151 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":71,"threads_started":1}
I20260812 16:32:37.851878 28541 heartbeater.cc:347] Connected to a master server at 127.19.238.124:39739
I20260812 16:32:37.851935 28541 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:37.852013 28541 heartbeater.cc:511] Master 127.19.238.124:39739 requested a full tablet report, sending...
I20260812 16:32:37.852416 28328 ts_manager.cc:194] Registered new tserver with Master: c8c477cc39284473822fb1229d47a24e (127.19.238.65:43035)
W20260812 16:32:37.938570 28548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:37.938796 28548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:37.938822 28548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:37.940021 28548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:37.940060 28548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:37.940088 28548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:32:37.941547 28548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:42945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.66.keytab
--principal=oryx/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:36503
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=44269
--tserver_master_addrs=127.19.238.126:46705,127.19.238.125:34857,127.19.238.124:39739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:37.941745 28548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:37.941923 28548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:37.947005 28554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:37.947017 28553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:37.947012 28556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:37.947407 28548 server_base.cc:1034] running on GCE node
I20260812 16:32:37.947660 28548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:37.947973 28548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:37.949148 28548 hybrid_clock.cc:648] HybridClock initialized: now 1786552357949130 us; error 46 us; skew 500 ppm
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552357, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:37.951771 28548 init.cc:377] Logged in from keytab as oryx/127.19.238.66@KRBTEST.COM (short username oryx)
I20260812 16:32:37.952280 28548 webserver.cc:460] Webserver started at http://127.19.238.66:44269/ using document root <none> and password file <none>
I20260812 16:32:37.952529 28548 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:37.952633 28548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:37.954154 28548 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:32:37.954990 28566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:37.955178 28548 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:37.955284 28548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "bec516f368e74cef842a68aad5c20bda"
format_stamp: "Formatted at 2026-08-12 16:32:36 on dist-test-slave-mvvj"
server_key: "e315600c9b409b9b642beb896665eac4"
server_key_iv: "fe8db5a08edfdba8240de2840d349b68"
server_key_version: "encryptionkey@0"
I20260812 16:32:37.955678 28548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:37.963341 28548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:37.965649 28548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:37.965813 28548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:37.966070 28548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:37.966401 28548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:37.966456 28548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:37.966499 28548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:37.966526 28548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:37.973066 28548 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:36503
I20260812 16:32:37.973109 28693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:36503 every 8 connection(s)
I20260812 16:32:37.973490 28548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:32:37.977299 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 28548
I20260812 16:32:37.977488 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 27795
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552357, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552357, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.125@KRBTEST.COM
I20260812 16:32:37.988193 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.974520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:42891 (local address 127.19.238.126:46705)
0812 16:32:37.974586 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.974589 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.974837 (+ 248us) server_negotiation.cc:407] Connection header received
0812 16:32:37.975767 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.975769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.975783 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.975818 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.976583 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.977366 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.980302 (+ 2936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.980638 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.980677 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.984321 (+ 3644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.984323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.984324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.984332 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.986613 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.986988 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.986989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.986989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.987044 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.987341 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.987342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.987342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.987482 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.987537 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.987974 (+ 437us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.988069 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 16:32:37.988365 28156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.974524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:38723 (local address 127.19.238.125:34857)
0812 16:32:37.974636 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.974638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.974651 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:32:37.975736 (+ 1085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.975737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.975751 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.975810 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.976569 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.977110 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.980301 (+ 3191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.980626 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.980672 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.984314 (+ 3642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.984316 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.984317 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.984327 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.986369 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.986947 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.986948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.986949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.987013 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.987330 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.987331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.987331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.987482 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.987537 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.988183 (+ 646us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.988270 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 16:32:37.988936 28699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.974261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.66:42891)
0812 16:32:37.974789 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.974798 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.975650 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.975941 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.975942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.976301 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.976471 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.976478 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.977492 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.977493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.980187 (+ 2694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.980190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.980761 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.980762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.980830 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.981736 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.981748 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.984178 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.986699 (+ 2521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.986700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.986701 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.986906 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.987113 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.987114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.987114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.987243 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.987577 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.987586 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.987777 (+ 191us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.988357 (+ 580us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.988670 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":1772,"thread_start_us":75,"threads_started":1}
I20260812 16:32:37.988936 28698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.974261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:34857 (local address 127.19.238.66:38723)
0812 16:32:37.974464 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.974513 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.975622 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.975928 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.975938 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.976252 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.976470 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.976474 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.977256 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.977257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.980187 (+ 2930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.980190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.980760 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.980761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.980832 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.981736 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.981748 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.984178 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.986478 (+ 2300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.986487 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.986490 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.986847 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.987089 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.987090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.987090 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.987243 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.987577 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.987586 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.987777 (+ 191us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.988445 (+ 668us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.988718 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":1907,"thread_start_us":95,"threads_started":1}
I20260812 16:32:37.989447 28694 heartbeater.cc:347] Connected to a master server at 127.19.238.126:46705
I20260812 16:32:37.989548 28694 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:37.989702 28694 heartbeater.cc:511] Master 127.19.238.126:46705 requested a full tablet report, sending...
I20260812 16:32:37.989900 28696 heartbeater.cc:347] Connected to a master server at 127.19.238.125:34857
I20260812 16:32:37.989946 28696 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:37.990015 28696 heartbeater.cc:511] Master 127.19.238.125:34857 requested a full tablet report, sending...
I20260812 16:32:37.990072 27408 ts_manager.cc:194] Re-registered known tserver with Master: bec516f368e74cef842a68aad5c20bda (127.19.238.66:36503)
I20260812 16:32:37.990310 28115 ts_manager.cc:194] Registered new tserver with Master: bec516f368e74cef842a68aad5c20bda (127.19.238.66:36503)
I20260812 16:32:37.990548 27408 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:42891
Aug 12 16:32:37 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552357, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.124@KRBTEST.COM
I20260812 16:32:37.995272 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.974262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:39739 (local address 127.19.238.66:49009)
0812 16:32:37.974661 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:37.974674 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:37.975650 (+ 976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:37.975940 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:37.975941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:37.976306 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:37.976471 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.976478 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.977520 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.977520 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.978623 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:37.978626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.979054 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.979054 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:37.979161 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.990166 (+ 11005us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:37.990173 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:37.991976 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:37.994008 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.994009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.994009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.994191 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.994448 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:37.994450 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:37.994450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:37.994531 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:37.994848 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:37.994849 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:37.994935 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 16:32:37.995013 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 16:32:37.995139 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":14,"thread_start_us":90,"threads_started":1}
I20260812 16:32:37.995298 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:37.974599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:49009 (local address 127.19.238.124:39739)
0812 16:32:37.974696 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:37.974698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:37.974754 (+ 56us) server_negotiation.cc:407] Connection header received
0812 16:32:37.975767 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:37.975769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:37.975786 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:37.975818 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:37.976589 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.977366 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.978718 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:37.978941 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:37.978976 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:37.992100 (+ 13124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:37.992102 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:37.992103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:37.992111 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:37.993927 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.994276 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.994278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.994278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.994356 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:37.994611 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:37.994612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:37.994613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:37.994771 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:37.994873 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:37.995054 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 16:32:37.995159 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 16:32:37.996132 28695 heartbeater.cc:347] Connected to a master server at 127.19.238.124:39739
I20260812 16:32:37.996207 28695 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:37.996294 28695 heartbeater.cc:511] Master 127.19.238.124:39739 requested a full tablet report, sending...
I20260812 16:32:37.996640 28328 ts_manager.cc:194] Registered new tserver with Master: bec516f368e74cef842a68aad5c20bda (127.19.238.66:36503)
I20260812 16:32:37.996942 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:42721
--local_ip_for_outbound_sockets=127.19.238.67
--tserver_master_addrs=127.19.238.126:46705
--webserver_port=41229
--webserver_interface=127.19.238.67
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.67.keytab
--principal=oryx/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:42945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.238.126:46705,127.19.238.125:34857
--tserver_master_addrs=127.19.238.126:46705,127.19.238.125:34857,127.19.238.124:39739 with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:38.082659 28703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:38.082893 28703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:38.082934 28703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:38.084133 28703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:38.084200 28703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:38.084246 28703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:32:38.085867 28703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:42945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.67.keytab
--principal=oryx/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:42721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=41229
--tserver_master_addrs=127.19.238.126:46705,127.19.238.125:34857,127.19.238.124:39739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:38.086133 28703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:38.086386 28703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:38.092142 28709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:38.092131 28711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:38.092135 28713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:38.092531 28703 server_base.cc:1034] running on GCE node
I20260812 16:32:38.092859 28703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:38.093129 28703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:38.094326 28703 hybrid_clock.cc:648] HybridClock initialized: now 1786552358094305 us; error 61 us; skew 500 ppm
Aug 12 16:32:38 dist-test-slave-mvvj krb5kdc[27359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552358, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:38.096990 28703 init.cc:377] Logged in from keytab as oryx/127.19.238.67@KRBTEST.COM (short username oryx)
I20260812 16:32:38.097462 28703 webserver.cc:460] Webserver started at http://127.19.238.67:41229/ using document root <none> and password file <none>
I20260812 16:32:38.097720 28703 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:38.097811 28703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:38.099193 28703 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:38.099802 28722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:38.099996 28703 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:38.100093 28703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "a60ad8f19246497b85c74a6f7d5d2695"
format_stamp: "Formatted at 2026-08-12 16:32:37 on dist-test-slave-mvvj"
server_key: "a3d5e7ff1af25eb264c4a6f59d75c6fc"
server_key_iv: "5c5e7a0b3d2d628981610b7d599b94af"
server_key_version: "encryptionkey@0"
I20260812 16:32:38.100454 28703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:38.115998 28703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:38.118090 28703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:38.118259 28703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:38.118515 28703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:38.118844 28703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:38.118897 28703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:38.119076 28703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:38.119119 28703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:38.125824 28703 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:42721
I20260812 16:32:38.125905 28854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:42721 every 8 connection(s)
I20260812 16:32:38.126384 28703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:32:38.133113 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 28703
Aug 12 16:32:38 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552358, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.125@KRBTEST.COM
Aug 12 16:32:38 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552358, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.124@KRBTEST.COM
Aug 12 16:32:38 dist-test-slave-mvvj krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552358, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM
I20260812 16:32:38.138993 28861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.127158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:39739 (local address 127.19.238.67:33653)
0812 16:32:38.127732 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:38.127742 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:38.128727 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:38.129065 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:38.129066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:38.129517 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:38.129711 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:38.129716 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.130702 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.130703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:38.131724 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:38.131726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.132269 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.132270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:38.132339 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.132742 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:38.132748 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:38.135165 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:38.137253 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:38.137261 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:38.137264 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:38.137590 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:38.137800 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:38.137801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:38.137801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:38.137932 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:38.138230 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:38.138235 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:38.138413 (+ 178us) client_negotiation.cc:769] Sending connection context
0812 16:32:38.138530 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 16:32:38.138708 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":62,"thread_start_us":115,"threads_started":1}
I20260812 16:32:38.139348 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.127160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:33653 (local address 127.19.238.124:39739)
0812 16:32:38.127226 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:38.127228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:38.127794 (+ 566us) server_negotiation.cc:407] Connection header received
0812 16:32:38.128859 (+ 1065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:38.128861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:38.128902 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:38.128960 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:38.129824 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.130598 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.131817 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.132113 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.132183 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.135293 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:38.135295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:38.135295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:38.135302 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:38.137159 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:38.137669 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:38.137670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:38.137670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:38.137732 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:38.138021 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:38.138022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:38.138023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:38.138148 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:38.138205 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:38.139159 (+ 954us) server_negotiation.cc:299] Negotiation successful
0812 16:32:38.139244 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 16:32:38.139343 28860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.127516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.67:50733)
0812 16:32:38.127899 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:38.127916 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:38.128754 (+ 838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:38.129065 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:38.129066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:38.129515 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:38.129711 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:38.129716 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.130702 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.130703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:38.131683 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:38.131685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.132275 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.132276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:38.132343 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.132939 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:38.132944 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:38.135627 (+ 2683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:38.138223 (+ 2596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:38.138224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:38.138224 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:38.138447 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:38.138652 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:38.138653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:38.138653 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:38.138709 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:38.138979 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:38.138980 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:38.139023 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 16:32:38.139103 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 16:32:38.139213 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":54,"thread_start_us":76,"threads_started":1}
I20260812 16:32:38.140033 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.127615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:50733 (local address 127.19.238.126:46705)
0812 16:32:38.127672 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:38.127674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:38.127970 (+ 296us) server_negotiation.cc:407] Connection header received
0812 16:32:38.128939 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:38.128941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:38.128955 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:38.129011 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:38.129826 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.130598 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.131823 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.132171 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.132273 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.135727 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:38.135729 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:38.135729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:38.135736 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:38.138145 (+ 2409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:38.138531 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:38.138533 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:38.138533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:38.138590 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:38.138764 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:38.138765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:38.138765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:38.138893 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:38.138952 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:38.139812 (+ 860us) server_negotiation.cc:299] Negotiation successful
0812 16:32:38.139935 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 16:32:38.140081 28856 heartbeater.cc:347] Connected to a master server at 127.19.238.124:39739
I20260812 16:32:38.140136 28156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.127139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:32785 (local address 127.19.238.125:34857)
0812 16:32:38.127193 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:38.127195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:38.127585 (+ 390us) server_negotiation.cc:407] Connection header received
0812 16:32:38.128859 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:38.128861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:38.128876 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:38.128907 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:38.129813 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.130315 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.131383 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.131594 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.131676 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.136094 (+ 4418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:38.136096 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:38.136096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:38.136104 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:38.138015 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:38.139046 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:38.139047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:38.139048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:38.139108 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:38.139477 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:38.139478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:38.139478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:38.139641 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:38.139691 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:38.139964 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 16:32:38.140040 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:32:38.140179 28856 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:38.140452 28856 heartbeater.cc:511] Master 127.19.238.124:39739 requested a full tablet report, sending...
I20260812 16:32:38.140466 28855 heartbeater.cc:347] Connected to a master server at 127.19.238.126:46705
I20260812 16:32:38.140523 28855 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:38.140583 28855 heartbeater.cc:511] Master 127.19.238.126:46705 requested a full tablet report, sending...
I20260812 16:32:38.140856 27408 ts_manager.cc:194] Re-registered known tserver with Master: a60ad8f19246497b85c74a6f7d5d2695 (127.19.238.67:42721)
I20260812 16:32:38.140933 28328 ts_manager.cc:194] Registered new tserver with Master: a60ad8f19246497b85c74a6f7d5d2695 (127.19.238.67:42721)
I20260812 16:32:38.141327 27408 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:50733
I20260812 16:32:38.141446 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.127158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:34857 (local address 127.19.238.67:32785)
0812 16:32:38.127357 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:38.127489 (+ 132us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:38.128726 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:38.129057 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:38.129062 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:38.129514 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:38.129710 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:38.129715 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.130436 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.130437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:38.131288 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:38.131290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.131732 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.131733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:38.131939 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.132584 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:38.132592 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:38.135985 (+ 3393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:38.138114 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:38.138116 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:38.138116 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:38.138961 (+ 845us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:38.139329 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:38.139330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:38.139330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:38.139396 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:38.139748 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:38.139748 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:38.139792 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 16:32:38.140336 (+ 544us) client_negotiation.cc:240] Negotiation successful
0812 16:32:38.141316 (+ 980us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":112,"threads_started":1}
I20260812 16:32:38.141759 28857 heartbeater.cc:347] Connected to a master server at 127.19.238.125:34857
I20260812 16:32:38.141811 28857 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:38.142077 28857 heartbeater.cc:511] Master 127.19.238.125:34857 requested a full tablet report, sending...
I20260812 16:32:38.142370 28115 ts_manager.cc:194] Registered new tserver with Master: a60ad8f19246497b85c74a6f7d5d2695 (127.19.238.67:42721)
I20260812 16:32:38.145560 28867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.134133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41056 (local address 127.19.238.126:46705)
0812 16:32:38.134589 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:38.134591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:38.134649 (+ 58us) server_negotiation.cc:407] Connection header received
0812 16:32:38.134770 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:38.134771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:38.134788 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:38.135200 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:38.136588 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.137395 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.138506 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.138810 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.142360 (+ 3550us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.142411 (+ 51us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:38.142412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:38.142412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:38.142417 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:38.144314 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:38.144738 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:38.144739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:38.144739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:38.144800 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:38.145018 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:38.145018 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:38.145019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:38.145139 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:38.145254 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:38.145345 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 16:32:38.145428 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":86,"threads_started":1}
I20260812 16:32:38.147082 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.134298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40152 (local address 127.19.238.125:34857)
0812 16:32:38.135594 (+ 1296us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:38.135596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:38.135606 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:38.135629 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:38.135630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:38.135642 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:38.135710 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:38.139900 (+ 4190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.140681 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.141718 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.142081 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.142143 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.143647 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:38.143648 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:38.143649 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:38.143656 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:38.145845 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:38.146178 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:38.146179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:38.146179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:38.146241 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:38.146458 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:38.146458 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:38.146459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:38.146619 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:38.146724 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:38.146870 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 16:32:38.146959 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1274,"thread_start_us":56,"threads_started":1}
I20260812 16:32:38.148722 28869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.134309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34958 (local address 127.19.238.124:39739)
0812 16:32:38.134810 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:38.134811 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:38.134909 (+ 98us) server_negotiation.cc:407] Connection header received
0812 16:32:38.135006 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:38.135007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:38.135020 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:38.135059 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:38.136576 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.138794 (+ 2218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.143053 (+ 4259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.143367 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.143583 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.145271 (+ 1688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:38.145272 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:38.145273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:38.145280 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:38.147418 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:38.147791 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:38.147792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:38.147793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:38.147891 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:38.148104 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:38.148105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:38.148105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:38.148231 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:38.148336 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:38.148436 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 16:32:38.148563 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":68,"threads_started":1}
I20260812 16:32:38.147534 27408 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:32:38.149492 27408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:32:38.157613 28884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.153763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48840 (local address 127.19.238.66:36503)
0812 16:32:38.153924 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:38.153928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:38.153948 (+ 20us) server_negotiation.cc:407] Connection header received
0812 16:32:38.153993 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:38.153994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:38.154011 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:38.154106 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:38.154208 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.155354 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.156764 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.157229 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.157318 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.157419 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:38.157460 (+ 41us) server_negotiation.cc:299] Negotiation successful
0812 16:32:38.157497 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
I20260812 16:32:38.157629 28288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.153405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:36503 (local address 127.0.0.1:48840)
0812 16:32:38.153561 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:38.153569 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:38.153602 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:38.154111 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:38.154112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:38.154114 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:38.154148 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:38.154149 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.155469 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.155470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:38.156672 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:38.156674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.157319 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.157320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:38.157382 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.157386 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:38.157429 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 16:32:38.157484 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146}
I20260812 16:32:38.157639 28067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.153298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:42721 (local address 127.0.0.1:43230)
0812 16:32:38.153456 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:38.153467 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:38.153506 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:38.154176 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:38.154177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:38.154179 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:38.154221 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:38.154223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.155546 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.155547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:38.156400 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:38.156403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.157361 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.157363 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:38.157431 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.157434 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:32:38.157464 (+ 30us) client_negotiation.cc:240] Negotiation successful
0812 16:32:38.157525 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132}
I20260812 16:32:38.157927 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.153386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43230 (local address 127.19.238.67:42721)
0812 16:32:38.154009 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:38.154013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:38.154022 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:32:38.154059 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:38.154061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:38.154076 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:38.154292 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:38.154350 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.155465 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.156497 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.157249 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.157679 (+ 430us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.157768 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:38.157800 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 16:32:38.157852 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":97,"threads_started":1}
I20260812 16:32:38.158372 28289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.153277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:43035 (local address 127.0.0.1:59954)
0812 16:32:38.153340 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:38.153351 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:38.153425 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:38.153983 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:38.153985 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:38.153987 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:38.154039 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:38.154041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.155283 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.155284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:38.156554 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:38.156557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.157419 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.157420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:38.157478 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.157482 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:38.157511 (+ 29us) client_negotiation.cc:240] Negotiation successful
0812 16:32:38.157552 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":83}
I20260812 16:32:38.158485 28620 tablet_service.cc:1467] Processing CreateTablet for tablet c90a756af2c642b5a8e7a3e70e858918 (DEFAULT_TABLE table=test-table [id=979f4feafb194281838af7f1a919475d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:38.158851 28620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c90a756af2c642b5a8e7a3e70e858918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:38.158731 28778 tablet_service.cc:1467] Processing CreateTablet for tablet c90a756af2c642b5a8e7a3e70e858918 (DEFAULT_TABLE table=test-table [id=979f4feafb194281838af7f1a919475d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:38.159090 28778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c90a756af2c642b5a8e7a3e70e858918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:38.159108 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.153597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59954 (local address 127.19.238.65:43035)
0812 16:32:38.153782 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:38.153788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:38.153799 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:32:38.153843 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:38.153845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:38.153864 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:38.154154 (+ 290us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:38.154227 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.155155 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.156647 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.157332 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.157632 (+ 300us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.158782 (+ 1150us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:38.158815 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 16:32:38.158867 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":93,"threads_started":1}
I20260812 16:32:38.160036 28460 tablet_service.cc:1467] Processing CreateTablet for tablet c90a756af2c642b5a8e7a3e70e858918 (DEFAULT_TABLE table=test-table [id=979f4feafb194281838af7f1a919475d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:38.160411 28460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c90a756af2c642b5a8e7a3e70e858918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:38.162237 28885 tablet_bootstrap.cc:492] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695: Bootstrap starting.
I20260812 16:32:38.162576 28886 tablet_bootstrap.cc:492] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda: Bootstrap starting.
I20260812 16:32:38.162945 28885 tablet_bootstrap.cc:654] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:38.163233 28886 tablet_bootstrap.cc:654] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:38.163256 28885 log.cc:824] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:38.163573 28886 log.cc:824] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:38.164131 28885 tablet_bootstrap.cc:492] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695: No bootstrap required, opened a new log
I20260812 16:32:38.164278 28885 ts_tablet_manager.cc:1397] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:32:38.164326 28886 tablet_bootstrap.cc:492] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda: No bootstrap required, opened a new log
I20260812 16:32:38.164340 28889 tablet_bootstrap.cc:492] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e: Bootstrap starting.
I20260812 16:32:38.164383 28886 ts_tablet_manager.cc:1397] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:32:38.164889 28889 tablet_bootstrap.cc:654] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:38.165199 28889 log.cc:824] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:38.165858 28889 tablet_bootstrap.cc:492] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e: No bootstrap required, opened a new log
I20260812 16:32:38.165937 28889 ts_tablet_manager.cc:1397] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:32:38.166507 28885 raft_consensus.cc:348] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } } peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } }
I20260812 16:32:38.166734 28885 raft_consensus.cc:374] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:38.166769 28885 raft_consensus.cc:729] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a60ad8f19246497b85c74a6f7d5d2695, State: Initialized, Role: FOLLOWER
I20260812 16:32:38.166915 28885 consensus_queue.cc:260] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } } peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } }
I20260812 16:32:38.166910 28886 raft_consensus.cc:348] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } } peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } }
I20260812 16:32:38.167047 28886 raft_consensus.cc:374] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:38.167097 28886 raft_consensus.cc:729] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bec516f368e74cef842a68aad5c20bda, State: Initialized, Role: FOLLOWER
I20260812 16:32:38.167218 28885 ts_tablet_manager.cc:1428] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:32:38.167240 28886 consensus_queue.cc:260] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } } peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } }
I20260812 16:32:38.167335 28855 heartbeater.cc:503] Master 127.19.238.126:46705 was elected leader, sending a full tablet report...
I20260812 16:32:38.167505 28886 ts_tablet_manager.cc:1428] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:32:38.167531 28694 heartbeater.cc:503] Master 127.19.238.126:46705 was elected leader, sending a full tablet report...
I20260812 16:32:38.168758 28889 raft_consensus.cc:348] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } } peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } }
I20260812 16:32:38.168919 28889 raft_consensus.cc:374] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:38.168962 28889 raft_consensus.cc:729] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8c477cc39284473822fb1229d47a24e, State: Initialized, Role: FOLLOWER
I20260812 16:32:38.169090 28889 consensus_queue.cc:260] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } } peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } }
I20260812 16:32:38.169382 28889 ts_tablet_manager.cc:1428] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:32:38.169452 28540 heartbeater.cc:503] Master 127.19.238.126:46705 was elected leader, sending a full tablet report...
I20260812 16:32:38.179738 28289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.176923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:39739 (local address 127.0.0.1:34974)
0812 16:32:38.176984 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:38.176999 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:38.177068 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:38.177210 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:38.177211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:38.177212 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:38.177260 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:38.177262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.177965 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.177966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:38.178759 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:38.178762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.179511 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.179513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:38.179568 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.179572 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:38.179631 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 16:32:38.179675 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 16:32:38.179765 28869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.177015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34974 (local address 127.19.238.124:39739)
0812 16:32:38.177075 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:38.177077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:38.177087 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:38.177116 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:38.177117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:38.177138 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:38.177198 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:38.177317 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.177880 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.178859 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.179263 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.179441 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.179505 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:38.179664 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 16:32:38.179712 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 16:32:38.180218 28338 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260812 16:32:38.181114 28373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9b8e1b2644347ccbb22cfc7f9622c8f. Latest consensus state: current_term: 1 leader_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: NON_VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: true } } }
I20260812 16:32:38.181224 28373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [sys.catalog]: This master's current role is: LEARNER
I20260812 16:32:38.216070 28897 raft_consensus.cc:482] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:32:38.216202 28897 raft_consensus.cc:504] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } } peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } }
I20260812 16:32:38.216778 28897 leader_election.cc:288] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a60ad8f19246497b85c74a6f7d5d2695 (127.19.238.67:42721), bec516f368e74cef842a68aad5c20bda (127.19.238.66:36503)
I20260812 16:32:38.221043 28884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.217248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:48295 (local address 127.19.238.66:36503)
0812 16:32:38.217319 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:38.217321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:38.217331 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:38.217358 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:38.217359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:38.217368 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:38.217429 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:38.217605 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.218382 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.220027 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.220602 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.220687 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.220696 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:38.220935 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 16:32:38.220976 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 16:32:38.221043 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.217115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:39401 (local address 127.19.238.67:42721)
0812 16:32:38.217191 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:38.217193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:38.217202 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:32:38.217232 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:38.217233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:38.217242 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:38.217318 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:38.217617 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.218451 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.220026 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.220598 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.220687 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.220696 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:38.220923 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 16:32:38.220968 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 16:32:38.221045 28547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.217024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:42721 (local address 127.19.238.65:39401)
0812 16:32:38.217076 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:38.217088 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:38.217158 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:38.217394 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:38.217395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:38.217396 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:38.217498 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:38.217500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.218545 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.218546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:38.219914 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:38.219917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.220716 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.220717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:38.220829 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.220833 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:38.220897 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 16:32:38.220946 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":109}
I20260812 16:32:38.221060 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.217140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:36503 (local address 127.19.238.65:48295)
0812 16:32:38.217187 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:38.217198 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:38.217237 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:38.217445 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:38.217445 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:38.217446 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:38.217499 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:38.217500 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.218547 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.218548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:38.219914 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:38.219917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.220709 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.220711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:38.220829 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.220833 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:38.220897 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 16:32:38.220946 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":45}
I20260812 16:32:38.221786 28642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c90a756af2c642b5a8e7a3e70e858918" candidate_uuid: "c8c477cc39284473822fb1229d47a24e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bec516f368e74cef842a68aad5c20bda" is_pre_election: true
I20260812 16:32:38.221958 28642 raft_consensus.cc:2454] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8c477cc39284473822fb1229d47a24e in term 0.
I20260812 16:32:38.221993 28791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c90a756af2c642b5a8e7a3e70e858918" candidate_uuid: "c8c477cc39284473822fb1229d47a24e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a60ad8f19246497b85c74a6f7d5d2695" is_pre_election: true
I20260812 16:32:38.222150 28791 raft_consensus.cc:2454] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8c477cc39284473822fb1229d47a24e in term 0.
I20260812 16:32:38.222247 28406 leader_election.cc:302] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bec516f368e74cef842a68aad5c20bda, c8c477cc39284473822fb1229d47a24e; no voters:
I20260812 16:32:38.222406 28897 raft_consensus.cc:2781] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:32:38.222466 28897 raft_consensus.cc:482] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:32:38.222489 28897 raft_consensus.cc:3037] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:38.223208 28897 raft_consensus.cc:504] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } } peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } }
I20260812 16:32:38.223372 28897 leader_election.cc:288] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [CANDIDATE]: Term 1 election: Requested vote from peers a60ad8f19246497b85c74a6f7d5d2695 (127.19.238.67:42721), bec516f368e74cef842a68aad5c20bda (127.19.238.66:36503)
I20260812 16:32:38.223531 28791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c90a756af2c642b5a8e7a3e70e858918" candidate_uuid: "c8c477cc39284473822fb1229d47a24e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a60ad8f19246497b85c74a6f7d5d2695"
I20260812 16:32:38.223580 28642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c90a756af2c642b5a8e7a3e70e858918" candidate_uuid: "c8c477cc39284473822fb1229d47a24e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bec516f368e74cef842a68aad5c20bda"
I20260812 16:32:38.223619 28791 raft_consensus.cc:3037] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:38.223651 28642 raft_consensus.cc:3037] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda [term 0 FOLLOWER]: Advancing to term 1
W20260812 16:32:38.224244 28697 tablet.cc:2369] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:32:38.224572 28791 raft_consensus.cc:2454] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8c477cc39284473822fb1229d47a24e in term 1.
I20260812 16:32:38.224572 28642 raft_consensus.cc:2454] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8c477cc39284473822fb1229d47a24e in term 1.
I20260812 16:32:38.224747 28406 leader_election.cc:302] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a60ad8f19246497b85c74a6f7d5d2695, c8c477cc39284473822fb1229d47a24e; no voters:
I20260812 16:32:38.224853 28897 raft_consensus.cc:2781] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:38.225028 28897 raft_consensus.cc:686] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [term 1 LEADER]: Becoming Leader. State: Replica: c8c477cc39284473822fb1229d47a24e, State: Running, Role: LEADER
I20260812 16:32:38.225143 28897 consensus_queue.cc:237] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } } peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } }
I20260812 16:32:38.225993 27408 catalog_manager.cc:5495] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e reported cstate change: term changed from 0 to 1, leader changed from <none> to c8c477cc39284473822fb1229d47a24e (127.19.238.65). New cstate: current_term: 1 leader_uuid: "c8c477cc39284473822fb1229d47a24e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } health_report { overall_health: UNKNOWN } } }
I20260812 16:32:38.226594 28069 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: NON_VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: true } } }
I20260812 16:32:38.226698 28069 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:38.227284 28287 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f: attempting to promote NON_VOTER 18228093a7184cbdb85bb16d8e59efa6 to VOTER
I20260812 16:32:38.227535 28287 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } }
I20260812 16:32:38.227948 28126 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 1 FOLLOWER]: Refusing update from remote peer a9b8e1b2644347ccbb22cfc7f9622c8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260812 16:32:38.228142 28337 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 1 LEARNER]: Refusing update from remote peer a9b8e1b2644347ccbb22cfc7f9622c8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260812 16:32:38.228303 28905 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260812 16:32:38.228389 28069 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260812 16:32:38.228991 28069 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 18228093a7184cbdb85bb16d8e59efa6 (127.19.238.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } } }
I20260812 16:32:38.229146 28126 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 18228093a7184cbdb85bb16d8e59efa6 (127.19.238.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } } }
I20260812 16:32:38.229214 28337 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 18228093a7184cbdb85bb16d8e59efa6 (127.19.238.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } } }
I20260812 16:32:38.229898 28069 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } } }
I20260812 16:32:38.229991 28069 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:38.230052 28069 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } } }
I20260812 16:32:38.230098 28068 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } } }
I20260812 16:32:38.230186 28068 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:38.230250 28373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } } }
I20260812 16:32:38.230335 28373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:32:38.230113 28069 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:38.230643 28906 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } } }
I20260812 16:32:38.230754 28906 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:32:38.235308 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.232207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59968 (local address 127.19.238.65:43035)
0812 16:32:38.232247 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:38.232249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:38.232259 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:38.232319 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:38.232320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:38.232336 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:38.232411 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:38.232485 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.232937 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.233728 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.233963 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.234031 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.234605 (+ 574us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:38.234743 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:38.234840 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:38.235212 (+ 372us) server_negotiation.cc:299] Negotiation successful
0812 16:32:38.235235 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:32:38.237524 28642 raft_consensus.cc:1261] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda [term 1 FOLLOWER]: Refusing update from remote peer c8c477cc39284473822fb1229d47a24e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:38.237620 28791 raft_consensus.cc:1261] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695 [term 1 FOLLOWER]: Refusing update from remote peer c8c477cc39284473822fb1229d47a24e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:38.237922 28902 consensus_queue.cc:1035] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:38.238091 28897 consensus_queue.cc:1035] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:38.241142 28908 mvcc.cc:205] Tried to move back new op lower bound from 7317718459337547776 to 7317718459290746880. Current Snapshot: MvccSnapshot[applied={T|T < 7317718459337547776}]
I20260812 16:32:38.242319 28909 mvcc.cc:205] Tried to move back new op lower bound from 7317718459337547776 to 7317718459290746880. Current Snapshot: MvccSnapshot[applied={T|T < 7317718459337547776}]
I20260812 16:32:38.242635 28910 mvcc.cc:205] Tried to move back new op lower bound from 7317718459337547776 to 7317718459290746880. Current Snapshot: MvccSnapshot[applied={T|T < 7317718459337547776}]
I20260812 16:32:38.244889 27407 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:38.244989 27407 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:38.246335 27407 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9b8e1b2644347ccbb22cfc7f9622c8f: Sending DeleteTablet for 3 replicas of tablet c90a756af2c642b5a8e7a3e70e858918
I20260812 16:32:38.246666 28778 tablet_service.cc:1514] Processing DeleteTablet for tablet c90a756af2c642b5a8e7a3e70e858918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:38 UTC) from {username='oryx', principal='oryx/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:43230
I20260812 16:32:38.246686 28620 tablet_service.cc:1514] Processing DeleteTablet for tablet c90a756af2c642b5a8e7a3e70e858918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:38 UTC) from {username='oryx', principal='oryx/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:48840
I20260812 16:32:38.246688 28460 tablet_service.cc:1514] Processing DeleteTablet for tablet c90a756af2c642b5a8e7a3e70e858918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:38 UTC) from {username='oryx', principal='oryx/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:59954
I20260812 16:32:38.246965 28918 tablet_replica.cc:330] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e: stopping tablet replica
I20260812 16:32:38.247072 28918 raft_consensus.cc:2229] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:32:38.247205 28918 raft_consensus.cc:2258] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:38.247383 28920 tablet_replica.cc:330] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695: stopping tablet replica
I20260812 16:32:38.247478 28920 raft_consensus.cc:2229] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:38.247582 28920 raft_consensus.cc:2258] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:38.248039 28918 ts_tablet_manager.cc:1905] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:38.248718 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 27372
I20260812 16:32:38.252004 28918 ts_tablet_manager.cc:1918] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:32:38.252254 28918 log.cc:1196] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-0/wal/wals/c90a756af2c642b5a8e7a3e70e858918
I20260812 16:32:38.252558 28918 ts_tablet_manager.cc:1939] T c90a756af2c642b5a8e7a3e70e858918 P c8c477cc39284473822fb1229d47a24e: Deleting consensus metadata
I20260812 16:32:38.253271 28920 ts_tablet_manager.cc:1905] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:38.254750 28917 tablet_replica.cc:330] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda: stopping tablet replica
I20260812 16:32:38.254894 28917 raft_consensus.cc:2229] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:38.255015 28917 raft_consensus.cc:2258] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:38.255209 28920 ts_tablet_manager.cc:1918] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:32:38.255316 28920 log.cc:1196] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/c90a756af2c642b5a8e7a3e70e858918
I20260812 16:32:38.255573 28920 ts_tablet_manager.cc:1939] T c90a756af2c642b5a8e7a3e70e858918 P a60ad8f19246497b85c74a6f7d5d2695: Deleting consensus metadata
I20260812 16:32:38.256268 28917 ts_tablet_manager.cc:1905] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:38.257752 28917 ts_tablet_manager.cc:1918] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:32:38.257858 28917 log.cc:1196] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/c90a756af2c642b5a8e7a3e70e858918
I20260812 16:32:38.258133 28917 ts_tablet_manager.cc:1939] T c90a756af2c642b5a8e7a3e70e858918 P bec516f368e74cef842a68aad5c20bda: Deleting consensus metadata
W20260812 16:32:38.259665 28727 connection.cc:535] server connection from 127.0.0.1:43230 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 16:32:38.259680 28570 connection.cc:535] server connection from 127.0.0.1:48840 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 16:32:38.259930 28406 connection.cc:535] server connection from 127.0.0.1:59954 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 16:32:38.268999 28869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.247614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34988 (local address 127.19.238.124:39739)
0812 16:32:38.247653 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:38.247655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:38.247663 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:32:38.247686 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:38.247687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:38.247697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:38.247756 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:38.261384 (+ 13628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.262153 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.263038 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.263357 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.263424 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.265085 (+ 1661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:38.265086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:38.265086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:38.265094 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:38.267194 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:38.268171 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:38.268173 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:38.268173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:38.268229 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:38.268456 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:38.268457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:38.268457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:38.268577 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:38.268687 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:38.268797 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 16:32:38.268879 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:32:38.270933 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.262225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40164 (local address 127.19.238.125:34857)
0812 16:32:38.262280 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:38.262282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:38.263124 (+ 842us) server_negotiation.cc:407] Connection header received
0812 16:32:38.263222 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:38.263223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:38.263238 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:38.263268 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:38.264381 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.265127 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.266153 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.266487 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.266530 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.268075 (+ 1545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:38.268077 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:38.268077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:38.268084 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:38.269787 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:38.270129 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:38.270130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:38.270131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:38.270194 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:38.270404 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:38.270405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:38.270406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:38.270525 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:38.270632 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:38.270744 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 16:32:38.270833 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 16:32:38.271762 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:38.261872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35002 (local address 127.19.238.124:39739)
0812 16:32:38.261912 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:38.261914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:38.261921 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:32:38.261945 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:38.261946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:38.261956 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:38.262017 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:38.262790 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.263483 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.264590 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:38.264898 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:38.264937 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:38.266449 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:38.266451 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:38.266451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:38.266459 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:38.270510 (+ 4051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:38.270955 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:38.270956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:38.270956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:38.271012 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:38.271227 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:38.271228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:38.271228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:38.271347 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:38.271416 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:38.271639 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 16:32:38.271704 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:32:39.243005 28930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:39.242665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.66:33577)
0812 16:32:39.242804 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:39.242936 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
W20260812 16:32:39.243216 28694 heartbeater.cc:644] Failed to heartbeat to 127.19.238.126:46705 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
I20260812 16:32:39.243489 28931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:39.242814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.65:35149)
0812 16:32:39.243365 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:39.243448 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":56,"threads_started":1}
I20260812 16:32:39.243489 28932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:39.242983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.67:33927)
0812 16:32:39.243382 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:39.243448 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":65,"threads_started":1}
W20260812 16:32:39.243654 28540 heartbeater.cc:644] Failed to heartbeat to 127.19.238.126:46705 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
W20260812 16:32:39.243806 28855 heartbeater.cc:644] Failed to heartbeat to 127.19.238.126:46705 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
I20260812 16:32:39.747465 28933 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a9b8e1b2644347ccbb22cfc7f9622c8f)
I20260812 16:32:39.747802 28934 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a9b8e1b2644347ccbb22cfc7f9622c8f)
I20260812 16:32:39.748000 28934 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } }
I20260812 16:32:39.748314 28934 leader_election.cc:288] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9b8e1b2644347ccbb22cfc7f9622c8f (127.19.238.126:46705), e4f1d932a8304008a7162555424cafbe (127.19.238.125:34857)
I20260812 16:32:39.747618 28933 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } }
I20260812 16:32:39.748827 28933 leader_election.cc:288] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9b8e1b2644347ccbb22cfc7f9622c8f (127.19.238.126:46705), 18228093a7184cbdb85bb16d8e59efa6 (127.19.238.124:39739)
I20260812 16:32:39.748857 28935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:39.748446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:41550)
0812 16:32:39.748760 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:39.748816 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":51,"threads_started":1}
W20260812 16:32:39.749071 28313 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
I20260812 16:32:39.749130 28938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:39.748720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:41564)
0812 16:32:39.749022 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:39.749075 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":45,"threads_started":1}
W20260812 16:32:39.749235 28100 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
I20260812 16:32:39.749612 28935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:39.749514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:41580)
0812 16:32:39.749556 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:39.749584 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 16:32:39.749642 28938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:39.749557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:41590)
0812 16:32:39.749594 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:39.749617 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 16:32:39.749717 28100 leader_election.cc:334] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a9b8e1b2644347ccbb22cfc7f9622c8f (127.19.238.126:46705): Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
W20260812 16:32:39.749920 28313 leader_election.cc:334] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a9b8e1b2644347ccbb22cfc7f9622c8f (127.19.238.126:46705): Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
I20260812 16:32:39.752739 28936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:39.748497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:34857 (local address 127.0.0.1:40178)
0812 16:32:39.748820 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:39.748832 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:39.749435 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:39.749662 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:39.749663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:39.749664 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:39.749719 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:39.749721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:39.750446 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:39.750446 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:39.751624 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:39.751627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:39.752456 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:39.752458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:39.752541 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:39.752546 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:32:39.752615 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 16:32:39.752666 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":55,"threads_started":1}
I20260812 16:32:39.752750 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:39.748935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:39739 (local address 127.0.0.1:35010)
0812 16:32:39.749080 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:39.749092 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:39.749151 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:39.749365 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:39.749367 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:39.749370 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:39.749432 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:39.749434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:39.750310 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:39.750312 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:39.751578 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:39.751581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:39.752509 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:39.752510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:39.752577 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:39.752581 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:39.752638 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 16:32:39.752683 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":41,"threads_started":1}
I20260812 16:32:39.752887 28941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:39.749074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35010 (local address 127.19.238.124:39739)
0812 16:32:39.749168 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:39.749170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:39.749179 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:32:39.749210 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:39.749211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:39.749222 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:39.749301 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:39.749522 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:39.750225 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:39.751729 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:39.752346 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:39.752440 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:39.752449 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:39.752700 (+ 251us) server_negotiation.cc:299] Negotiation successful
0812 16:32:39.752750 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":79,"thread_start_us":54,"threads_started":1}
I20260812 16:32:39.752996 28937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:39.748655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40178 (local address 127.19.238.125:34857)
0812 16:32:39.748757 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:39.748759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:39.749406 (+ 647us) server_negotiation.cc:407] Connection header received
0812 16:32:39.749527 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:39.749528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:39.749538 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:39.749591 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:39.749796 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:39.750367 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:39.751729 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:39.752369 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:39.752440 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:39.752449 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:39.752685 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 16:32:39.752743 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":53,"threads_started":1}
I20260812 16:32:39.753122 28337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e4f1d932a8304008a7162555424cafbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "18228093a7184cbdb85bb16d8e59efa6" is_pre_election: true
I20260812 16:32:39.753206 28126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "18228093a7184cbdb85bb16d8e59efa6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e4f1d932a8304008a7162555424cafbe" is_pre_election: true
I20260812 16:32:39.753244 28337 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4f1d932a8304008a7162555424cafbe in term 1.
I20260812 16:32:39.753299 28126 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18228093a7184cbdb85bb16d8e59efa6 in term 1.
I20260812 16:32:39.753430 28099 leader_election.cc:302] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18228093a7184cbdb85bb16d8e59efa6, e4f1d932a8304008a7162555424cafbe; no voters: a9b8e1b2644347ccbb22cfc7f9622c8f
I20260812 16:32:39.753477 28314 leader_election.cc:302] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18228093a7184cbdb85bb16d8e59efa6, e4f1d932a8304008a7162555424cafbe; no voters: a9b8e1b2644347ccbb22cfc7f9622c8f
I20260812 16:32:39.753544 28933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 16:32:39.753576 28934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 16:32:39.753590 28933 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 1 FOLLOWER]: Starting leader election (detected failure of leader a9b8e1b2644347ccbb22cfc7f9622c8f)
I20260812 16:32:39.753613 28933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 1 FOLLOWER]: Advancing to term 2
I20260812 16:32:39.753613 28934 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a9b8e1b2644347ccbb22cfc7f9622c8f)
I20260812 16:32:39.753635 28934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 1 FOLLOWER]: Advancing to term 2
I20260812 16:32:39.754616 28933 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } }
I20260812 16:32:39.754660 28934 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } }
I20260812 16:32:39.754768 28933 leader_election.cc:288] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [CANDIDATE]: Term 2 election: Requested vote from peers a9b8e1b2644347ccbb22cfc7f9622c8f (127.19.238.126:46705), 18228093a7184cbdb85bb16d8e59efa6 (127.19.238.124:39739)
I20260812 16:32:39.754776 28934 leader_election.cc:288] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [CANDIDATE]: Term 2 election: Requested vote from peers a9b8e1b2644347ccbb22cfc7f9622c8f (127.19.238.126:46705), e4f1d932a8304008a7162555424cafbe (127.19.238.125:34857)
I20260812 16:32:39.754957 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:39.754860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:41602)
0812 16:32:39.754895 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:39.754927 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260812 16:32:39.755079 28126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "18228093a7184cbdb85bb16d8e59efa6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e4f1d932a8304008a7162555424cafbe"
I20260812 16:32:39.755100 28337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e4f1d932a8304008a7162555424cafbe" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "18228093a7184cbdb85bb16d8e59efa6"
I20260812 16:32:39.755192 28126 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 18228093a7184cbdb85bb16d8e59efa6 in current term 2: Already voted for candidate e4f1d932a8304008a7162555424cafbe in this term.
I20260812 16:32:39.755193 28337 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e4f1d932a8304008a7162555424cafbe in current term 2: Already voted for candidate 18228093a7184cbdb85bb16d8e59efa6 in this term.
I20260812 16:32:39.755313 28936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:39.754934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:41606)
0812 16:32:39.754971 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:39.755289 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260812 16:32:39.755451 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:39.755246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:41620)
0812 16:32:39.755402 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:39.755430 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
W20260812 16:32:39.755532 28100 leader_election.cc:334] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a9b8e1b2644347ccbb22cfc7f9622c8f (127.19.238.126:46705): Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
I20260812 16:32:39.755565 28100 leader_election.cc:302] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4f1d932a8304008a7162555424cafbe; no voters: 18228093a7184cbdb85bb16d8e59efa6, a9b8e1b2644347ccbb22cfc7f9622c8f
I20260812 16:32:39.755657 28936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:39.755557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:41630)
0812 16:32:39.755609 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:39.755638 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 16:32:39.755716 28933 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260812 16:32:39.755722 28313 leader_election.cc:334] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a9b8e1b2644347ccbb22cfc7f9622c8f (127.19.238.126:46705): Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
I20260812 16:32:39.755802 28313 leader_election.cc:302] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18228093a7184cbdb85bb16d8e59efa6; no voters: a9b8e1b2644347ccbb22cfc7f9622c8f, e4f1d932a8304008a7162555424cafbe
I20260812 16:32:39.755901 28934 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 16:32:40.244077 28944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:40.243602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.66:58705)
0812 16:32:40.243852 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:40.243925 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":96,"threads_started":1}
I20260812 16:32:40.244356 28945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:40.244002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.65:56081)
0812 16:32:40.244266 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:40.244307 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":70,"threads_started":1}
I20260812 16:32:40.244522 28946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:40.244129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.67:33149)
0812 16:32:40.244433 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:40.244474 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":69,"threads_started":1}
I20260812 16:32:41.245059 28947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.244562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.66:34549)
0812 16:32:41.244950 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.244999 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":92,"threads_started":1}
I20260812 16:32:41.245337 28949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.244987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.65:47823)
0812 16:32:41.245245 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.245286 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":109,"threads_started":1}
I20260812 16:32:41.245376 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.245199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.67:60749)
0812 16:32:41.245305 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.245331 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":54,"threads_started":1}
I20260812 16:32:41.275457 28952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:32:41.275573 28952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } }
I20260812 16:32:41.275754 28952 leader_election.cc:288] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a9b8e1b2644347ccbb22cfc7f9622c8f (127.19.238.126:46705), e4f1d932a8304008a7162555424cafbe (127.19.238.125:34857)
I20260812 16:32:41.276106 28126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "18228093a7184cbdb85bb16d8e59efa6" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e4f1d932a8304008a7162555424cafbe" is_pre_election: true
I20260812 16:32:41.276185 28953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.275946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:41860)
0812 16:32:41.276077 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.276127 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
I20260812 16:32:41.276232 28126 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18228093a7184cbdb85bb16d8e59efa6 in term 2.
I20260812 16:32:41.276413 28314 leader_election.cc:302] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18228093a7184cbdb85bb16d8e59efa6, e4f1d932a8304008a7162555424cafbe; no voters:
I20260812 16:32:41.276499 28952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 16:32:41.276543 28952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:32:41.276575 28952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 2 FOLLOWER]: Advancing to term 3
I20260812 16:32:41.276714 28953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.276616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:41866)
0812 16:32:41.276652 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.276680 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260812 16:32:41.276837 28313 leader_election.cc:334] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a9b8e1b2644347ccbb22cfc7f9622c8f (127.19.238.126:46705): Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
I20260812 16:32:41.277456 28952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } }
I20260812 16:32:41.277623 28952 leader_election.cc:288] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [CANDIDATE]: Term 3 election: Requested vote from peers a9b8e1b2644347ccbb22cfc7f9622c8f (127.19.238.126:46705), e4f1d932a8304008a7162555424cafbe (127.19.238.125:34857)
I20260812 16:32:41.277792 28126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "18228093a7184cbdb85bb16d8e59efa6" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e4f1d932a8304008a7162555424cafbe"
I20260812 16:32:41.277840 28953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.277730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:41872)
0812 16:32:41.277790 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.277813 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 16:32:41.277873 28126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 2 FOLLOWER]: Advancing to term 3
I20260812 16:32:41.278196 28953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.278102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:41880)
0812 16:32:41.278154 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.278180 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 16:32:41.278311 28313 leader_election.cc:334] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a9b8e1b2644347ccbb22cfc7f9622c8f (127.19.238.126:46705): Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
I20260812 16:32:41.278723 28126 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18228093a7184cbdb85bb16d8e59efa6 in term 3.
I20260812 16:32:41.278893 28314 leader_election.cc:302] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18228093a7184cbdb85bb16d8e59efa6, e4f1d932a8304008a7162555424cafbe; no voters: a9b8e1b2644347ccbb22cfc7f9622c8f
I20260812 16:32:41.278990 28952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 16:32:41.279101 28952 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [term 3 LEADER]: Becoming Leader. State: Replica: 18228093a7184cbdb85bb16d8e59efa6, State: Running, Role: LEADER
I20260812 16:32:41.279191 28952 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } }
I20260812 16:32:41.279585 28956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18228093a7184cbdb85bb16d8e59efa6. Latest consensus state: current_term: 3 leader_uuid: "18228093a7184cbdb85bb16d8e59efa6" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } } }
I20260812 16:32:41.279702 28956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:41.279929 28959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:32:41.280339 28959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:32:41.280398 28959 catalog_manager.cc:1259] Loaded cluster ID: e2c224ef7aa047f6b3d0c62416405557
I20260812 16:32:41.280448 28959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:32:41.280642 28959 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:41.280731 28959 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: Loaded TSK: 0
I20260812 16:32:41.280877 28959 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:32:41.294747 28328 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:32:41.296541 28328 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:32:41.297784 28953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.297621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:41884)
0812 16:32:41.297701 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.297744 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 16:32:41.297887 28126 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [term 3 FOLLOWER]: Refusing update from remote peer 18228093a7184cbdb85bb16d8e59efa6: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260812 16:32:41.298207 28953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.298080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.0.0.1:41886)
0812 16:32:41.298129 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.298154 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 16:32:41.298151 28952 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260812 16:32:41.298344 28313 consensus_peers.cc:487] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 -> Peer a9b8e1b2644347ccbb22cfc7f9622c8f (127.19.238.126:46705): Couldn't send request to peer a9b8e1b2644347ccbb22cfc7f9622c8f. Status: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 16:32:41.298916 28961 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18228093a7184cbdb85bb16d8e59efa6. Latest consensus state: current_term: 3 leader_uuid: "18228093a7184cbdb85bb16d8e59efa6" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } } }
I20260812 16:32:41.299067 28961 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:32:41.299436 28960 mvcc.cc:205] Tried to move back new op lower bound from 7317718471872745472 to 7317718471800086528. Current Snapshot: MvccSnapshot[applied={T|T < 7317718471872745472}]
I20260812 16:32:41.299546 28956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "18228093a7184cbdb85bb16d8e59efa6" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } } }
I20260812 16:32:41.299640 28956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6 [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:41.299913 28961 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "18228093a7184cbdb85bb16d8e59efa6" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a9b8e1b2644347ccbb22cfc7f9622c8f" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46705 } } peers { permanent_uuid: "e4f1d932a8304008a7162555424cafbe" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 34857 } attrs { promote: false } } peers { permanent_uuid: "18228093a7184cbdb85bb16d8e59efa6" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 39739 } attrs { promote: false } } }
I20260812 16:32:41.300011 28961 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4f1d932a8304008a7162555424cafbe [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:32:41.305433 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.301961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59980 (local address 127.19.238.65:43035)
0812 16:32:41.302103 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:41.302105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:41.302112 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:32:41.302143 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:41.302144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:41.302155 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:41.302232 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:41.302373 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.303161 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.304436 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.305032 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.305104 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:41.305116 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:41.305290 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 16:32:41.305350 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260812 16:32:41.305462 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.301768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:43035 (local address 127.0.0.1:59980)
0812 16:32:41.301942 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.301954 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:41.302007 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:41.302237 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:41.302238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:41.302240 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:41.302300 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:41.302302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.303252 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.303253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:41.304352 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:41.304355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.305152 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.305153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:41.305221 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:41.305226 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:32:41.305282 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 16:32:41.305326 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":2,"thread_start_us":85,"threads_started":1}
I20260812 16:32:41.305662 28953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.301710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:42721 (local address 127.0.0.1:43242)
0812 16:32:41.301744 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.301756 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:41.301820 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:41.302230 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:41.302232 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:41.302233 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:41.302299 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:41.302301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.303220 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.303221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:41.304480 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:41.304483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.305311 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.305312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:41.305363 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:41.305367 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:41.305398 (+ 31us) client_negotiation.cc:240] Negotiation successful
0812 16:32:41.305440 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":41}
I20260812 16:32:41.305630 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.301935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48844 (local address 127.19.238.66:36503)
0812 16:32:41.302066 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:41.302069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:41.302409 (+ 340us) server_negotiation.cc:407] Connection header received
0812 16:32:41.302490 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:41.302491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:41.302504 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:41.302576 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:41.302726 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.303461 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.304728 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.305107 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.305193 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:41.305204 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:41.305498 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 16:32:41.305564 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":60,"threads_started":1}
I20260812 16:32:41.305467 28967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.301805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:36503 (local address 127.0.0.1:48844)
0812 16:32:41.302092 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.302102 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:41.302144 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:41.302587 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:41.302588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:41.302590 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:41.302627 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:41.302628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.303542 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.303543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:41.304626 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:41.304628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.305195 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.305196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:41.305259 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:41.305264 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:32:41.305317 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 16:32:41.305357 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":84,"threads_started":1}
I20260812 16:32:41.305701 28460 tablet_service.cc:1467] Processing CreateTablet for tablet f05794a27ec14b3fbd25d19103b20b5c (DEFAULT_TABLE table=test-table [id=7d04b4fe1f2e4a5ebae43d59e38dcd78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:41.305845 28460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f05794a27ec14b3fbd25d19103b20b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:41.306203 28966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.301774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43242 (local address 127.19.238.67:42721)
0812 16:32:41.302073 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:41.302075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:41.302084 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:32:41.302115 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:41.302116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:41.302127 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:41.302369 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:41.302415 (+ 46us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.303121 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.304575 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.305163 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.306054 (+ 891us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:41.306069 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:41.306090 (+ 21us) server_negotiation.cc:299] Negotiation successful
0812 16:32:41.306133 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":53,"threads_started":1}
I20260812 16:32:41.306385 28778 tablet_service.cc:1467] Processing CreateTablet for tablet f05794a27ec14b3fbd25d19103b20b5c (DEFAULT_TABLE table=test-table [id=7d04b4fe1f2e4a5ebae43d59e38dcd78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:41.306501 28778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f05794a27ec14b3fbd25d19103b20b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:41.307214 28971 tablet_bootstrap.cc:492] T f05794a27ec14b3fbd25d19103b20b5c P c8c477cc39284473822fb1229d47a24e: Bootstrap starting.
I20260812 16:32:41.307298 28620 tablet_service.cc:1467] Processing CreateTablet for tablet f05794a27ec14b3fbd25d19103b20b5c (DEFAULT_TABLE table=test-table [id=7d04b4fe1f2e4a5ebae43d59e38dcd78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:41.307425 28620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f05794a27ec14b3fbd25d19103b20b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:41.307639 28971 tablet_bootstrap.cc:654] T f05794a27ec14b3fbd25d19103b20b5c P c8c477cc39284473822fb1229d47a24e: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:41.307904 28972 tablet_bootstrap.cc:492] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695: Bootstrap starting.
I20260812 16:32:41.308318 28972 tablet_bootstrap.cc:654] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:41.308377 28971 tablet_bootstrap.cc:492] T f05794a27ec14b3fbd25d19103b20b5c P c8c477cc39284473822fb1229d47a24e: No bootstrap required, opened a new log
I20260812 16:32:41.308472 28971 ts_tablet_manager.cc:1397] T f05794a27ec14b3fbd25d19103b20b5c P c8c477cc39284473822fb1229d47a24e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:41.308611 28971 raft_consensus.cc:348] T f05794a27ec14b3fbd25d19103b20b5c P c8c477cc39284473822fb1229d47a24e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } } peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } }
I20260812 16:32:41.308698 28971 raft_consensus.cc:374] T f05794a27ec14b3fbd25d19103b20b5c P c8c477cc39284473822fb1229d47a24e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:41.308727 28971 raft_consensus.cc:729] T f05794a27ec14b3fbd25d19103b20b5c P c8c477cc39284473822fb1229d47a24e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8c477cc39284473822fb1229d47a24e, State: Initialized, Role: FOLLOWER
I20260812 16:32:41.308775 28971 consensus_queue.cc:260] T f05794a27ec14b3fbd25d19103b20b5c P c8c477cc39284473822fb1229d47a24e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } } peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } }
I20260812 16:32:41.308996 28971 ts_tablet_manager.cc:1428] T f05794a27ec14b3fbd25d19103b20b5c P c8c477cc39284473822fb1229d47a24e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 16:32:41.309003 28974 tablet_bootstrap.cc:492] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda: Bootstrap starting.
I20260812 16:32:41.309388 28974 tablet_bootstrap.cc:654] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:41.309473 28949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.309354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.65:38269)
0812 16:32:41.309400 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.309439 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 16:32:41.309810 28541 heartbeater.cc:503] Master 127.19.238.124:39739 was elected leader, sending a full tablet report...
I20260812 16:32:41.309994 28949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.309899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.65:43903)
0812 16:32:41.309939 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.309964 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 16:32:41.310235 28972 tablet_bootstrap.cc:492] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695: No bootstrap required, opened a new log
I20260812 16:32:41.310293 28972 ts_tablet_manager.cc:1397] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:32:41.310359 28974 tablet_bootstrap.cc:492] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda: No bootstrap required, opened a new log
I20260812 16:32:41.310420 28974 ts_tablet_manager.cc:1397] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:41.310422 28972 raft_consensus.cc:348] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } } peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } }
I20260812 16:32:41.310508 28972 raft_consensus.cc:374] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:41.310542 28972 raft_consensus.cc:729] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a60ad8f19246497b85c74a6f7d5d2695, State: Initialized, Role: FOLLOWER
I20260812 16:32:41.310550 28974 raft_consensus.cc:348] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } } peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } }
I20260812 16:32:41.310592 28972 consensus_queue.cc:260] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } } peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } }
I20260812 16:32:41.310632 28974 raft_consensus.cc:374] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:41.310662 28974 raft_consensus.cc:729] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bec516f368e74cef842a68aad5c20bda, State: Initialized, Role: FOLLOWER
I20260812 16:32:41.310706 28974 consensus_queue.cc:260] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } } peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } }
I20260812 16:32:41.310778 28972 ts_tablet_manager.cc:1428] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:41.310886 28974 ts_tablet_manager.cc:1428] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:41.311008 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.310905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.67:39425)
0812 16:32:41.310951 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.310977 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 16:32:41.311182 28947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.311085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.66:57389)
0812 16:32:41.311131 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.311156 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 16:32:41.311259 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.311183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.67:59383)
0812 16:32:41.311216 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.311237 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260812 16:32:41.311599 28947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.311504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.66:50255)
0812 16:32:41.311541 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.311567 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 16:32:41.311686 28695 heartbeater.cc:503] Master 127.19.238.124:39739 was elected leader, sending a full tablet report...
I20260812 16:32:41.311755 28856 heartbeater.cc:503] Master 127.19.238.124:39739 was elected leader, sending a full tablet report...
W20260812 16:32:41.334858 28543 tablet.cc:2369] T f05794a27ec14b3fbd25d19103b20b5c P c8c477cc39284473822fb1229d47a24e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:32:41.348664 28978 raft_consensus.cc:482] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:32:41.348757 28978 raft_consensus.cc:504] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } } peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } }
I20260812 16:32:41.349090 28978 leader_election.cc:288] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8c477cc39284473822fb1229d47a24e (127.19.238.65:43035), bec516f368e74cef842a68aad5c20bda (127.19.238.66:36503)
I20260812 16:32:41.352128 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.349404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:48927 (local address 127.19.238.66:36503)
0812 16:32:41.349466 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:41.349468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:41.349580 (+ 112us) server_negotiation.cc:407] Connection header received
0812 16:32:41.349620 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:41.349621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:41.349630 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:41.349682 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:41.349901 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.350393 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.351359 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.351724 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.351812 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:41.351821 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:41.352004 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 16:32:41.352044 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 16:32:41.352149 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.349239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:43035 (local address 127.19.238.67:59771)
0812 16:32:41.349285 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.349297 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:41.349359 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:41.349723 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:41.349724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:41.349727 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:41.349802 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:41.349803 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.350496 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.350497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:41.351274 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:41.351276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.351880 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.351881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:41.351938 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:41.351942 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:41.352020 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 16:32:41.352060 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 16:32:41.352149 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.349328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:59771 (local address 127.19.238.65:43035)
0812 16:32:41.349457 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:41.349460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:41.349469 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:32:41.349501 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:41.349502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:41.349512 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:41.349591 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:41.349890 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.350403 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.351370 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.351743 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.351832 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:41.351842 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:41.352032 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 16:32:41.352081 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260812 16:32:41.352185 28980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.349259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:36503 (local address 127.19.238.67:48927)
0812 16:32:41.349500 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.349510 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:41.349568 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:41.349713 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:41.349714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:41.349717 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:41.349801 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:41.349803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.350482 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.350483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:41.351264 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:41.351266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.351811 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.351812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:41.351911 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:41.351915 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:41.351992 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 16:32:41.352040 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":2,"thread_start_us":131,"threads_started":1}
I20260812 16:32:41.352469 28484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05794a27ec14b3fbd25d19103b20b5c" candidate_uuid: "a60ad8f19246497b85c74a6f7d5d2695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8c477cc39284473822fb1229d47a24e" is_pre_election: true
I20260812 16:32:41.352650 28484 raft_consensus.cc:2454] T f05794a27ec14b3fbd25d19103b20b5c P c8c477cc39284473822fb1229d47a24e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a60ad8f19246497b85c74a6f7d5d2695 in term 0.
I20260812 16:32:41.352609 28642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05794a27ec14b3fbd25d19103b20b5c" candidate_uuid: "a60ad8f19246497b85c74a6f7d5d2695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bec516f368e74cef842a68aad5c20bda" is_pre_election: true
I20260812 16:32:41.352720 28642 raft_consensus.cc:2454] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a60ad8f19246497b85c74a6f7d5d2695 in term 0.
I20260812 16:32:41.352841 28727 leader_election.cc:302] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a60ad8f19246497b85c74a6f7d5d2695, c8c477cc39284473822fb1229d47a24e; no voters:
I20260812 16:32:41.353010 28978 raft_consensus.cc:2781] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:32:41.353063 28978 raft_consensus.cc:482] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:32:41.353096 28978 raft_consensus.cc:3037] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:41.353888 28978 raft_consensus.cc:504] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } } peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } }
I20260812 16:32:41.354039 28978 leader_election.cc:288] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [CANDIDATE]: Term 1 election: Requested vote from peers c8c477cc39284473822fb1229d47a24e (127.19.238.65:43035), bec516f368e74cef842a68aad5c20bda (127.19.238.66:36503)
I20260812 16:32:41.354238 28642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05794a27ec14b3fbd25d19103b20b5c" candidate_uuid: "a60ad8f19246497b85c74a6f7d5d2695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bec516f368e74cef842a68aad5c20bda"
I20260812 16:32:41.354211 28484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05794a27ec14b3fbd25d19103b20b5c" candidate_uuid: "a60ad8f19246497b85c74a6f7d5d2695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8c477cc39284473822fb1229d47a24e"
I20260812 16:32:41.354327 28642 raft_consensus.cc:3037] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:41.354327 28484 raft_consensus.cc:3037] T f05794a27ec14b3fbd25d19103b20b5c P c8c477cc39284473822fb1229d47a24e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:41.355180 28642 raft_consensus.cc:2454] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a60ad8f19246497b85c74a6f7d5d2695 in term 1.
I20260812 16:32:41.355180 28484 raft_consensus.cc:2454] T f05794a27ec14b3fbd25d19103b20b5c P c8c477cc39284473822fb1229d47a24e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a60ad8f19246497b85c74a6f7d5d2695 in term 1.
I20260812 16:32:41.355402 28726 leader_election.cc:302] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a60ad8f19246497b85c74a6f7d5d2695, bec516f368e74cef842a68aad5c20bda; no voters:
I20260812 16:32:41.355521 28978 raft_consensus.cc:2781] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:41.355688 28978 raft_consensus.cc:686] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [term 1 LEADER]: Becoming Leader. State: Replica: a60ad8f19246497b85c74a6f7d5d2695, State: Running, Role: LEADER
I20260812 16:32:41.355773 28978 consensus_queue.cc:237] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } } peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } }
I20260812 16:32:41.356084 28980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.355976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.67:52521)
0812 16:32:41.356014 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.356053 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 16:32:41.356881 28327 catalog_manager.cc:5495] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 reported cstate change: term changed from 0 to 1, leader changed from <none> to a60ad8f19246497b85c74a6f7d5d2695 (127.19.238.67). New cstate: current_term: 1 leader_uuid: "a60ad8f19246497b85c74a6f7d5d2695" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a60ad8f19246497b85c74a6f7d5d2695" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 42721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 } health_report { overall_health: UNKNOWN } } }
I20260812 16:32:41.363996 28966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.361783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43254 (local address 127.19.238.67:42721)
0812 16:32:41.361820 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:41.361822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:41.361829 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:32:41.361883 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:41.361884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:41.361901 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:41.361965 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:41.362079 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.362584 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.363283 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:41.363463 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:41.363530 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:41.363670 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:41.363767 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:41.363818 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:41.363913 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 16:32:41.363938 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:32:41.365257 28642 raft_consensus.cc:1261] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda [term 1 FOLLOWER]: Refusing update from remote peer a60ad8f19246497b85c74a6f7d5d2695: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:41.365279 28484 raft_consensus.cc:1261] T f05794a27ec14b3fbd25d19103b20b5c P c8c477cc39284473822fb1229d47a24e [term 1 FOLLOWER]: Refusing update from remote peer a60ad8f19246497b85c74a6f7d5d2695: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:41.365629 28981 consensus_queue.cc:1035] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8c477cc39284473822fb1229d47a24e" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:41.365804 28978 consensus_queue.cc:1035] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bec516f368e74cef842a68aad5c20bda" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:41.367123 28947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.366915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.66:36259)
0812 16:32:41.367047 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.367088 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260812 16:32:41.367304 28949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.366926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.65:60507)
0812 16:32:41.367245 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.367274 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300}
I20260812 16:32:41.368775 28949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.368679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.65:41627)
0812 16:32:41.368719 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.368744 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 16:32:41.369392 28980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.369277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.67:40305)
0812 16:32:41.369332 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.369358 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 16:32:41.369808 28947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:41.369616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46705 (local address 127.19.238.66:47665)
0812 16:32:41.369745 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.369779 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260812 16:32:41.370587 28328 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:41.370684 28328 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:41.372149 28328 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18228093a7184cbdb85bb16d8e59efa6: Sending DeleteTablet for 3 replicas of tablet f05794a27ec14b3fbd25d19103b20b5c
I20260812 16:32:41.372584 28460 tablet_service.cc:1514] Processing DeleteTablet for tablet f05794a27ec14b3fbd25d19103b20b5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:41 UTC) from {username='oryx', principal='oryx/127.19.238.124@KRBTEST.COM'} at 127.0.0.1:59980
I20260812 16:32:41.372718 28778 tablet_service.cc:1514] Processing DeleteTablet for tablet f05794a27ec14b3fbd25d19103b20b5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:41 UTC) from {username='oryx', principal='oryx/127.19.238.124@KRBTEST.COM'} at 127.0.0.1:43242
I20260812 16:32:41.372776 28620 tablet_service.cc:1514] Processing DeleteTablet for tablet f05794a27ec14b3fbd25d19103b20b5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:41 UTC) from {username='oryx', principal='oryx/127.19.238.124@KRBTEST.COM'} at 127.0.0.1:48844
I20260812 16:32:41.372970 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 28283
I20260812 16:32:41.372964 28996 tablet_replica.cc:330] T f05794a27ec14b3fbd25d19103b20b5c P c8c477cc39284473822fb1229d47a24e: stopping tablet replica
I20260812 16:32:41.376456 28998 tablet_replica.cc:330] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda: stopping tablet replica
I20260812 16:32:41.376525 28998 raft_consensus.cc:2229] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:41.376576 28998 raft_consensus.cc:2258] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:41.377557 28998 ts_tablet_manager.cc:1905] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:41.379794 28998 ts_tablet_manager.cc:1918] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:32:41.379882 28998 log.cc:1196] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/f05794a27ec14b3fbd25d19103b20b5c
I20260812 16:32:41.380118 28998 ts_tablet_manager.cc:1939] T f05794a27ec14b3fbd25d19103b20b5c P bec516f368e74cef842a68aad5c20bda: Deleting consensus metadata
I20260812 16:32:41.380450 28312 catalog_manager.cc:4841] TS bec516f368e74cef842a68aad5c20bda (127.19.238.66:36503): tablet f05794a27ec14b3fbd25d19103b20b5c (table test-table [id=7d04b4fe1f2e4a5ebae43d59e38dcd78]) successfully deleted
I20260812 16:32:41.384575 28997 tablet_replica.cc:330] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695: stopping tablet replica
I20260812 16:32:41.385056 28997 raft_consensus.cc:2229] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:32:41.385203 28997 raft_consensus.cc:2258] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:41.385519 28997 ts_tablet_manager.cc:1905] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:41.387421 28997 ts_tablet_manager.cc:1918] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:32:41.387494 28997 log.cc:1196] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/f05794a27ec14b3fbd25d19103b20b5c
I20260812 16:32:41.387728 28997 ts_tablet_manager.cc:1939] T f05794a27ec14b3fbd25d19103b20b5c P a60ad8f19246497b85c74a6f7d5d2695: Deleting consensus metadata
I20260812 16:32:41.388195 28312 catalog_manager.cc:4841] TS a60ad8f19246497b85c74a6f7d5d2695 (127.19.238.67:42721): tablet f05794a27ec14b3fbd25d19103b20b5c (table test-table [id=7d04b4fe1f2e4a5ebae43d59e38dcd78]) successfully deleted
W20260812 16:32:41.390295 28967 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:41.390144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:43035 (local address 127.0.0.1:59986)
0812 16:32:41.390202 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:41.390229 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:43035: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 16:32:41.390439 28313 catalog_manager.cc:4584] TS c8c477cc39284473822fb1229d47a24e (127.19.238.65:43035): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f05794a27ec14b3fbd25d19103b20b5c: Network error: Client connection negotiation failed: client connection to 127.19.238.65:43035: connect: Connection reset by peer (error 104)
I20260812 16:32:41.390564 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 28548
I20260812 16:32:41.400928 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 28703
I20260812 16:32:41.407560 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 27952
I20260812 16:32:41.413966 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 28175
2026-08-12T16:32:41Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6261 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:32:41 dist-test-slave-mvvj krb5kdc[29001](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:32:41 dist-test-slave-mvvj krb5kdc[29001](info): set up 2 sockets
Aug 12 16:32:41 dist-test-slave-mvvj krb5kdc[29001](info): commencing operation
krb5kdc: starting...
W20260812 16:32:42.604918 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:32:42 dist-test-slave-mvvj krb5kdc[29001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:32:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:32:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.126.keytab.
Entry for principal oryx/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.126.keytab.
I20260812 16:32:42.734233 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:44561
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.126.keytab
--principal=oryx/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:45759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:44561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:42.818020 29017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:42.818240 29017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:42.818262 29017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:42.819380 29017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:42.819416 29017 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 16:32:42.819429 29017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:42.819442 29017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:42.819455 29017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:42.819466 29017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:32:42.821045 29017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:45759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:44561
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.126.keytab
--principal=oryx/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:44561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:42.821265 29017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:42.821465 29017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:42.826483 29026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:42.826498 29024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:42.826713 29017 server_base.cc:1034] running on GCE node
W20260812 16:32:42.826517 29023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:42.827054 29017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:42.827373 29017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:42.828563 29017 hybrid_clock.cc:648] HybridClock initialized: now 1786552362828543 us; error 61 us; skew 500 ppm
Aug 12 16:32:42 dist-test-slave-mvvj krb5kdc[29001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552362, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:42.831301 29017 init.cc:377] Logged in from keytab as oryx/127.19.238.126@KRBTEST.COM (short username oryx)
I20260812 16:32:42.831806 29017 webserver.cc:460] Webserver started at http://127.19.238.126:40125/ using document root <none> and password file <none>
I20260812 16:32:42.832129 29017 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:42.832204 29017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:42.832365 29017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:42.833462 29017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "40ba34de68d34a16ad5ef4404de1d00a"
format_stamp: "Formatted at 2026-08-12 16:32:42 on dist-test-slave-mvvj"
server_key: "d2de6995413d5f71c322ff283e42ce4c"
server_key_iv: "f763527e756f1c7c23a30596ffd39b89"
server_key_version: "encryptionkey@0"
I20260812 16:32:42.833853 29017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "40ba34de68d34a16ad5ef4404de1d00a"
format_stamp: "Formatted at 2026-08-12 16:32:42 on dist-test-slave-mvvj"
server_key: "d2de6995413d5f71c322ff283e42ce4c"
server_key_iv: "f763527e756f1c7c23a30596ffd39b89"
server_key_version: "encryptionkey@0"
I20260812 16:32:42.835287 29017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:32:42.836194 29036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:42.836410 29017 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:42.836524 29017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "40ba34de68d34a16ad5ef4404de1d00a"
format_stamp: "Formatted at 2026-08-12 16:32:42 on dist-test-slave-mvvj"
server_key: "d2de6995413d5f71c322ff283e42ce4c"
server_key_iv: "f763527e756f1c7c23a30596ffd39b89"
server_key_version: "encryptionkey@0"
I20260812 16:32:42.836627 29017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:42.848838 29017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:42.851315 29017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:42.851490 29017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:42.856498 29017 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:44561
I20260812 16:32:42.856617 29107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:44561 every 8 connection(s)
I20260812 16:32:42.856952 29017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:32:42.857682 29108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:42.859762 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 29017
I20260812 16:32:42.859903 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:32:42.860114 20409 external_mini_cluster.cc:1428] Setting key f8f443bf6b17755be908d5021468e466
I20260812 16:32:42.860435 29108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a: Bootstrap starting.
I20260812 16:32:42.861166 29108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:42.861479 29108 log.cc:824] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:42.862164 29108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a: No bootstrap required, opened a new log
I20260812 16:32:42.864182 29108 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40ba34de68d34a16ad5ef4404de1d00a" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 44561 } }
I20260812 16:32:42.864382 29108 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:42.864446 29108 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40ba34de68d34a16ad5ef4404de1d00a, State: Initialized, Role: FOLLOWER
I20260812 16:32:42.864625 29108 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40ba34de68d34a16ad5ef4404de1d00a" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 44561 } }
I20260812 16:32:42.864724 29108 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:32:42.864771 29108 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:32:42.864820 29108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:42.865628 29108 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40ba34de68d34a16ad5ef4404de1d00a" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 44561 } }
I20260812 16:32:42.865751 29108 leader_election.cc:302] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40ba34de68d34a16ad5ef4404de1d00a; no voters:
I20260812 16:32:42.865942 29108 leader_election.cc:288] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:32:42.866012 29114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:42.866183 29114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a [term 1 LEADER]: Becoming Leader. State: Replica: 40ba34de68d34a16ad5ef4404de1d00a, State: Running, Role: LEADER
I20260812 16:32:42.866256 29108 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:42.866331 29114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40ba34de68d34a16ad5ef4404de1d00a" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 44561 } }
I20260812 16:32:42.866676 29114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40ba34de68d34a16ad5ef4404de1d00a. Latest consensus state: current_term: 1 leader_uuid: "40ba34de68d34a16ad5ef4404de1d00a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40ba34de68d34a16ad5ef4404de1d00a" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 44561 } } }
I20260812 16:32:42.866761 29114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:42.866977 29119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 16:32:42 dist-test-slave-mvvj krb5kdc[29001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM
I20260812 16:32:42.867736 29116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40ba34de68d34a16ad5ef4404de1d00a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40ba34de68d34a16ad5ef4404de1d00a" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 44561 } } }
I20260812 16:32:42.867821 29116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:42.868201 29119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:32:42.870114 29119 catalog_manager.cc:1347] Generated new cluster ID: cb077ec911a44356908b9c1e3bf0322a
I20260812 16:32:42.870170 29119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:32:42.872100 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:42.860677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47080 (local address 127.19.238.126:44561)
0812 16:32:42.860876 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:42.860887 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:42.860924 (+ 37us) server_negotiation.cc:407] Connection header received
0812 16:32:42.861474 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:42.861482 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:42.861556 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:42.861726 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:42.862428 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:42.863314 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:42.864185 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:42.864793 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:42.864911 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:42.868013 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:42.868018 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:42.868019 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:42.868031 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:42.870149 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:42.870567 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:42.870570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:42.870570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:42.870670 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:42.870858 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:42.870858 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:42.870859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:42.871137 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:42.871281 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:42.871722 (+ 441us) server_negotiation.cc:299] Negotiation successful
0812 16:32:42.871883 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":88,"threads_started":1}
I20260812 16:32:42.881871 29119 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:32:42.882634 29119 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:42.885766 29119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a: Generated new TSK 0
I20260812 16:32:42.885969 29119 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.65.keytab.
Entry for principal oryx/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.65.keytab.
I20260812 16:32:42.919902 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:44561
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.65.keytab
--principal=oryx/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:45759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:43.036676 29137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:43.036900 29137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:43.036936 29137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:43.038323 29137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:43.038375 29137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:43.038399 29137 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:32:43.038420 29137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:32:43.040349 29137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:45759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.65.keytab
--principal=oryx/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.238.126:44561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:43.040668 29137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:43.040963 29137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:43.046538 29143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:43.046612 29137 server_base.cc:1034] running on GCE node
W20260812 16:32:43.046674 29142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:43.046538 29145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:43.047056 29137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:43.047299 29137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:43.048456 29137 hybrid_clock.cc:648] HybridClock initialized: now 1786552363048425 us; error 51 us; skew 500 ppm
Aug 12 16:32:43 dist-test-slave-mvvj krb5kdc[29001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552363, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:43.051199 29137 init.cc:377] Logged in from keytab as oryx/127.19.238.65@KRBTEST.COM (short username oryx)
I20260812 16:32:43.051754 29137 webserver.cc:460] Webserver started at http://127.19.238.65:39881/ using document root <none> and password file <none>
I20260812 16:32:43.052055 29137 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:43.052109 29137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:43.052294 29137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:43.053503 29137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "7d50f865d8894bb784ce0f6d7654011c"
format_stamp: "Formatted at 2026-08-12 16:32:43 on dist-test-slave-mvvj"
server_key: "a3d03ef3822e0dcb21aeab2b497f8cec"
server_key_iv: "d449bae72a1635c33e1bc6d9925bef48"
server_key_version: "encryptionkey@0"
I20260812 16:32:43.053915 29137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "7d50f865d8894bb784ce0f6d7654011c"
format_stamp: "Formatted at 2026-08-12 16:32:43 on dist-test-slave-mvvj"
server_key: "a3d03ef3822e0dcb21aeab2b497f8cec"
server_key_iv: "d449bae72a1635c33e1bc6d9925bef48"
server_key_version: "encryptionkey@0"
I20260812 16:32:43.055548 29137 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:32:43.056442 29153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:43.056624 29137 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:43.056754 29137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "7d50f865d8894bb784ce0f6d7654011c"
format_stamp: "Formatted at 2026-08-12 16:32:43 on dist-test-slave-mvvj"
server_key: "a3d03ef3822e0dcb21aeab2b497f8cec"
server_key_iv: "d449bae72a1635c33e1bc6d9925bef48"
server_key_version: "encryptionkey@0"
I20260812 16:32:43.056839 29137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:43.067420 29137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:43.069770 29137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:43.069945 29137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:43.070487 29137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:43.070531 29137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:43.070556 29137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:43.070569 29137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:43.077783 29137 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:41743
I20260812 16:32:43.077884 29295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:41743 every 8 connection(s)
I20260812 16:32:43.078255 29137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
Aug 12 16:32:43 dist-test-slave-mvvj krb5kdc[29001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552363, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM
I20260812 16:32:43.081663 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.071136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:48795 (local address 127.19.238.126:44561)
0812 16:32:43.071194 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.071196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.071441 (+ 245us) server_negotiation.cc:407] Connection header received
0812 16:32:43.072107 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.072109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.072123 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.072175 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:43.073126 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.074001 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.075629 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.076029 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.076125 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.079025 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:43.079027 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:43.079027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:43.079035 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:43.080362 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:43.080749 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:43.080750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:43.080750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:43.080817 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:43.081065 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:43.081067 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:43.081067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:43.081197 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:43.081266 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.081510 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.081595 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 16:32:43.081863 29164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.071137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:44561 (local address 127.19.238.65:48795)
0812 16:32:43.071316 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:43.071354 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:43.071987 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:43.072249 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:43.072254 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:43.072686 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:43.072991 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.072997 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.074116 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.074117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.075504 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.075507 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.076133 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.076134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.076311 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.076921 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:43.076933 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:43.078928 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:43.080439 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:43.080446 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:43.080447 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:43.080680 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:43.080878 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:43.080878 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:43.080878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:43.080977 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:43.081265 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:43.081270 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:43.081387 (+ 117us) client_negotiation.cc:769] Sending connection context
0812 16:32:43.081469 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 16:32:43.081615 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":98,"threads_started":1}
I20260812 16:32:43.082492 29296 heartbeater.cc:347] Connected to a master server at 127.19.238.126:44561
I20260812 16:32:43.082582 29296 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:43.082784 29296 heartbeater.cc:511] Master 127.19.238.126:44561 requested a full tablet report, sending...
I20260812 16:32:43.083369 29053 ts_manager.cc:194] Registered new tserver with Master: 7d50f865d8894bb784ce0f6d7654011c (127.19.238.65:41743)
I20260812 16:32:43.084306 29053 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:48795
I20260812 16:32:43.087378 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 29137
I20260812 16:32:43.087494 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:32:43.087771 20409 external_mini_cluster.cc:1428] Setting key 89fa14d9a80427e10b8481016355a6c6
I20260812 16:32:43.089588 29302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.083653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:44561 (local address 127.19.238.65:43639)
0812 16:32:43.083776 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:43.083787 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:43.083882 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:43.084013 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:43.084014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:43.084172 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:43.084222 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.084223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.085010 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.085011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.085552 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.085554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.085868 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.085869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.085917 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.086399 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:43.086407 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:43.086713 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:43.088465 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:43.088466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:43.088466 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:43.088680 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:43.088903 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:43.088942 (+ 39us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:43.088943 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:43.089031 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:43.089329 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:43.089329 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:43.089370 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 16:32:43.089423 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 16:32:43.089516 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
I20260812 16:32:43.089694 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.083725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:43639 (local address 127.19.238.126:44561)
0812 16:32:43.083898 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.083900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.083908 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:32:43.083931 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.083932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.083942 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.084016 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:43.084290 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.084908 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.085626 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.085802 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.085853 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.086791 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:43.086792 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:43.086792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:43.086797 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:43.088378 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:43.088756 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:43.088757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:43.088757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:43.088837 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:43.089112 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:43.089113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:43.089114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:43.089265 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:43.089334 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.089466 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.089558 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
WARNING: no policy specified for oryx/127.19.238.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.66.keytab.
Entry for principal oryx/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.66.keytab.
I20260812 16:32:43.115403 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:44561
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.66.keytab
--principal=oryx/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:45759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:43.200500 29305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:43.200730 29305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:43.200757 29305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:43.201946 29305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:43.201994 29305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:43.202015 29305 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:32:43.202034 29305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:32:43.203630 29305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:45759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.66.keytab
--principal=oryx/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.238.126:44561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:43.203931 29305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:43.204181 29305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:43.209357 29313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:43.209395 29310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:43.209410 29311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:43.209637 29305 server_base.cc:1034] running on GCE node
I20260812 16:32:43.209935 29305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:43.210202 29305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:43.211370 29305 hybrid_clock.cc:648] HybridClock initialized: now 1786552363211356 us; error 28 us; skew 500 ppm
Aug 12 16:32:43 dist-test-slave-mvvj krb5kdc[29001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552363, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:43.214020 29305 init.cc:377] Logged in from keytab as oryx/127.19.238.66@KRBTEST.COM (short username oryx)
I20260812 16:32:43.214498 29305 webserver.cc:460] Webserver started at http://127.19.238.66:34137/ using document root <none> and password file <none>
I20260812 16:32:43.214747 29305 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:43.214821 29305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:43.214975 29305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:43.216145 29305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "976cd6755f3a4a17ba2e220f591ea9f5"
format_stamp: "Formatted at 2026-08-12 16:32:43 on dist-test-slave-mvvj"
server_key: "19f6a5fc4bfad0cccd079920b984187e"
server_key_iv: "11571ec7bfe8e6d8bb842da7cf180ccc"
server_key_version: "encryptionkey@0"
I20260812 16:32:43.216526 29305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "976cd6755f3a4a17ba2e220f591ea9f5"
format_stamp: "Formatted at 2026-08-12 16:32:43 on dist-test-slave-mvvj"
server_key: "19f6a5fc4bfad0cccd079920b984187e"
server_key_iv: "11571ec7bfe8e6d8bb842da7cf180ccc"
server_key_version: "encryptionkey@0"
I20260812 16:32:43.217909 29305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 16:32:43.218731 29322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:43.218909 29305 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:43.219012 29305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "976cd6755f3a4a17ba2e220f591ea9f5"
format_stamp: "Formatted at 2026-08-12 16:32:43 on dist-test-slave-mvvj"
server_key: "19f6a5fc4bfad0cccd079920b984187e"
server_key_iv: "11571ec7bfe8e6d8bb842da7cf180ccc"
server_key_version: "encryptionkey@0"
I20260812 16:32:43.219110 29305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:43.236922 29305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:43.239004 29305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:43.239194 29305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:43.239928 29305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:43.239990 29305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:43.240023 29305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:43.240049 29305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 16:32:43 dist-test-slave-mvvj krb5kdc[29001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552363, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM
I20260812 16:32:43.247598 29305 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:46241
I20260812 16:32:43.248095 29305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:32:43.248456 29456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:46241 every 8 connection(s)
I20260812 16:32:43.249462 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.240322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:53195 (local address 127.19.238.126:44561)
0812 16:32:43.240374 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.240377 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.240654 (+ 277us) server_negotiation.cc:407] Connection header received
0812 16:32:43.241186 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.241188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.241203 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.241292 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:43.242018 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.242616 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.243627 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.243924 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.244006 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.246594 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:43.246595 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:43.246596 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:43.246604 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:43.248255 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:43.248753 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:43.248757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:43.248757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:43.248814 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:43.249006 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:43.249007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:43.249007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:43.249103 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:43.249146 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.249341 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.249400 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 16:32:43.249727 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.240338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:44561 (local address 127.19.238.66:53195)
0812 16:32:43.240515 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:43.240556 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:43.241077 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:43.241389 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:43.241393 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:43.241696 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:43.241890 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.241896 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.242720 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.242720 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.243522 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.243524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.244014 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.244015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.244142 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.244685 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:43.244693 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:43.246491 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:43.248345 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:43.248351 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:43.248353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:43.248682 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:43.248873 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:43.248874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:43.248874 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:43.248968 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:43.249143 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:43.249146 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:43.249256 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 16:32:43.249331 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 16:32:43.249450 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260812 16:32:43.250308 29458 heartbeater.cc:347] Connected to a master server at 127.19.238.126:44561
I20260812 16:32:43.250394 29458 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:43.250602 29458 heartbeater.cc:511] Master 127.19.238.126:44561 requested a full tablet report, sending...
I20260812 16:32:43.250984 29053 ts_manager.cc:194] Registered new tserver with Master: 976cd6755f3a4a17ba2e220f591ea9f5 (127.19.238.66:46241)
I20260812 16:32:43.251047 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 29305
I20260812 16:32:43.251168 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:32:43.251411 20409 external_mini_cluster.cc:1428] Setting key 33dc8fd661d0fae6e72db30a93ae3254
I20260812 16:32:43.251504 29053 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:53195
I20260812 16:32:43.257927 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.251433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:46165 (local address 127.19.238.126:44561)
0812 16:32:43.251467 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.251469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.251630 (+ 161us) server_negotiation.cc:407] Connection header received
0812 16:32:43.251933 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.251934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.251945 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.251998 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:43.252475 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.252922 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.253992 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.254204 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.254285 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.255225 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:43.255226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:43.255227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:43.255233 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:43.256717 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:43.257083 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:43.257084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:43.257085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:43.257142 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:43.257357 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:43.257358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:43.257359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:43.257511 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:43.257597 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.257754 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.257836 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:32:43.257939 29465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.251375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:44561 (local address 127.19.238.66:46165)
0812 16:32:43.251562 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:43.251573 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:43.251667 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:43.252042 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:43.252043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:43.252300 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:43.252382 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.252385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.253023 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.253024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.253877 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.253880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.254283 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.254284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.254348 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.254806 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:43.254811 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:43.255118 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:43.256823 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:43.256825 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:43.256825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:43.257003 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:43.257226 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:43.257226 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:43.257227 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:43.257293 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:43.257587 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:43.257588 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:43.257635 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 16:32:43.257725 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 16:32:43.257832 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for oryx/127.19.238.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.238.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.67.keytab.
Entry for principal oryx/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.67.keytab.
I20260812 16:32:43.279815 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:44561
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.67.keytab
--principal=oryx/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:45759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:43.364753 29469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:43.365036 29469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:43.365080 29469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:43.366300 29469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:43.366348 29469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:43.366374 29469 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:32:43.366402 29469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:32:43.368274 29469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:45759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.67.keytab
--principal=oryx/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.238.126:44561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:43.368527 29469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:43.368742 29469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:43.373952 29481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:43.373955 29476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:43.374024 29475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:43.374236 29469 server_base.cc:1034] running on GCE node
I20260812 16:32:43.374562 29469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:43.374830 29469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:43.376027 29469 hybrid_clock.cc:648] HybridClock initialized: now 1786552363376011 us; error 32 us; skew 500 ppm
Aug 12 16:32:43 dist-test-slave-mvvj krb5kdc[29001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552363, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:43.378676 29469 init.cc:377] Logged in from keytab as oryx/127.19.238.67@KRBTEST.COM (short username oryx)
I20260812 16:32:43.379173 29469 webserver.cc:460] Webserver started at http://127.19.238.67:36605/ using document root <none> and password file <none>
I20260812 16:32:43.379437 29469 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:43.379513 29469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:43.379666 29469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:43.380890 29469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "8585b614b6cc42c0b9cb217b143ae39f"
format_stamp: "Formatted at 2026-08-12 16:32:43 on dist-test-slave-mvvj"
server_key: "7f442270027dfa290a71b26a6535b795"
server_key_iv: "45d4d397dcbd06a496e96f0b0076e60d"
server_key_version: "encryptionkey@0"
I20260812 16:32:43.381276 29469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "8585b614b6cc42c0b9cb217b143ae39f"
format_stamp: "Formatted at 2026-08-12 16:32:43 on dist-test-slave-mvvj"
server_key: "7f442270027dfa290a71b26a6535b795"
server_key_iv: "45d4d397dcbd06a496e96f0b0076e60d"
server_key_version: "encryptionkey@0"
I20260812 16:32:43.382670 29469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:32:43.383451 29488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:43.383631 29469 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:43.383752 29469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "8585b614b6cc42c0b9cb217b143ae39f"
format_stamp: "Formatted at 2026-08-12 16:32:43 on dist-test-slave-mvvj"
server_key: "7f442270027dfa290a71b26a6535b795"
server_key_iv: "45d4d397dcbd06a496e96f0b0076e60d"
server_key_version: "encryptionkey@0"
I20260812 16:32:43.383872 29469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:43.400830 29469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:43.402791 29469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:43.402971 29469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:43.403546 29469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:43.403615 29469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:43.403661 29469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:43.403707 29469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 16:32:43 dist-test-slave-mvvj krb5kdc[29001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552363, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM
I20260812 16:32:43.411592 29469 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:35269
I20260812 16:32:43.411757 29622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:35269 every 8 connection(s)
I20260812 16:32:43.412119 29469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:32:43.413887 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.404145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:34595 (local address 127.19.238.126:44561)
0812 16:32:43.404216 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.404218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.404429 (+ 211us) server_negotiation.cc:407] Connection header received
0812 16:32:43.405300 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.405302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.405317 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.405385 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:43.406226 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.406818 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.407939 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.408146 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.408220 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.410859 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:43.410861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:43.410862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:43.410872 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:43.412679 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:43.413115 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:43.413116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:43.413116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:43.413171 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:43.413380 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:43.413381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:43.413381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:43.413509 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:43.413558 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.413763 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.413824 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 16:32:43.414152 29498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.404047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:44561 (local address 127.19.238.67:34595)
0812 16:32:43.404270 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:43.404323 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:43.405165 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:43.405521 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:43.405525 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:43.405917 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:43.406116 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.406123 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.406920 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.406921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.407808 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.407811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.408242 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.408243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.408366 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.408877 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:43.408885 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:43.410745 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:43.412780 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:43.412786 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:43.412787 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:43.413041 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:43.413234 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:43.413234 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:43.413234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:43.413326 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:43.413558 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:43.413561 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:43.413673 (+ 112us) client_negotiation.cc:769] Sending connection context
0812 16:32:43.413750 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 16:32:43.413869 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":107,"threads_started":1}
I20260812 16:32:43.414652 29624 heartbeater.cc:347] Connected to a master server at 127.19.238.126:44561
I20260812 16:32:43.414746 29624 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:43.414944 29624 heartbeater.cc:511] Master 127.19.238.126:44561 requested a full tablet report, sending...
I20260812 16:32:43.415352 29053 ts_manager.cc:194] Registered new tserver with Master: 8585b614b6cc42c0b9cb217b143ae39f (127.19.238.67:35269)
I20260812 16:32:43.415875 29053 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:34595
I20260812 16:32:43.415913 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 29469
I20260812 16:32:43.416004 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:32:43.416246 20409 external_mini_cluster.cc:1428] Setting key 556e085a2857d003205b98404f1f9dbf
I20260812 16:32:43.417750 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:32:43.422197 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.415770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:39749 (local address 127.19.238.126:44561)
0812 16:32:43.415809 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.415811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.415964 (+ 153us) server_negotiation.cc:407] Connection header received
0812 16:32:43.416277 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.416278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.416289 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.416335 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:43.416756 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.417252 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.418249 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.418494 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.418551 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.419570 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:43.419572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:43.419572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:43.419578 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:43.421007 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:43.421396 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:43.421397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:43.421397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:43.421491 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:43.421699 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:43.421700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:43.421700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:43.421795 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:43.421844 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.422015 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.422136 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:32:43.422289 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.415698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:44561 (local address 127.19.238.67:39749)
0812 16:32:43.415884 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:43.415894 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:43.415978 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:43.416350 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:43.416351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:43.416619 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:43.416685 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.416687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.417342 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.417342 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.418167 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.418171 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.418574 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.418575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.418643 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.419058 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:43.419064 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:43.419473 (+ 409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:43.421094 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:43.421095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:43.421096 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:43.421320 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:43.421559 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:43.421560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:43.421560 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:43.421633 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:43.421857 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:43.421858 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:43.421910 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 16:32:43.421998 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 16:32:43.422137 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":74,"threads_started":1}
Aug 12 16:32:43 dist-test-slave-mvvj krb5kdc[29001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552363, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 16:32:43 dist-test-slave-mvvj krb5kdc[29001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:43 dist-test-slave-mvvj krb5kdc[29001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 16:32:43.441375 29112 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:43.435454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47096 (local address 127.19.238.126:44561)
0812 16:32:43.435529 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.435531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.435539 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:32:43.435572 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.435573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.435582 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.435641 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:43.436301 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.436816 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.437567 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.437745 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.437840 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.441269 (+ 3429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47096: BlockingRecv error: recv got EOF from 127.0.0.1:47096 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 16:32:43 dist-test-slave-mvvj krb5kdc[29001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552363, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM
I20260812 16:32:43.449254 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.442251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47098 (local address 127.19.238.126:44561)
0812 16:32:43.442290 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.442292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.442416 (+ 124us) server_negotiation.cc:407] Connection header received
0812 16:32:43.442470 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.442470 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.442479 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.442527 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:43.442993 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.443462 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.444227 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.444404 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.444457 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.446652 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:43.446653 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:43.446653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:43.446660 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:43.448174 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:43.448595 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:43.448596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:43.448596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:43.448651 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:43.448848 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:43.448849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:43.448849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:43.448955 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:43.449029 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.449124 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.449186 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:32:43.451123 29053 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47098:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:32:43.452970 29053 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:32:43.459730 29654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.455483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:41743 (local address 127.0.0.1:35104)
0812 16:32:43.456174 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:43.456237 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:43.456304 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:43.456780 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:43.456781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:43.456783 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:43.456839 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.456841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.457706 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.457707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.458715 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.458717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.459491 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.459492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.459566 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.459579 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 16:32:43.459619 (+ 40us) client_negotiation.cc:240] Negotiation successful
0812 16:32:43.459658 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"mutex_wait_us":4,"thread_start_us":65,"threads_started":1}
I20260812 16:32:43.461198 29652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.455508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:35269 (local address 127.0.0.1:38062)
0812 16:32:43.456433 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:43.456443 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:43.456493 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:43.456699 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:43.456704 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:43.456707 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:43.456789 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.456791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.458472 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.458474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.459780 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.459783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.460885 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.460886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.460974 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.460978 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:43.461071 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 16:32:43.461116 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"thread_start_us":73,"threads_started":1}
I20260812 16:32:43.461232 29653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.455635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35104 (local address 127.19.238.65:41743)
0812 16:32:43.456634 (+ 999us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.456639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.456649 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:43.456686 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.456688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.456701 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.456768 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:43.456895 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.457613 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.458802 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.459325 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.460091 (+ 766us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.460194 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.460222 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.461132 (+ 910us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":958,"thread_start_us":104,"threads_started":1}
I20260812 16:32:43.461443 29655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.455750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38062 (local address 127.19.238.67:35269)
0812 16:32:43.455969 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.455974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.456493 (+ 519us) server_negotiation.cc:407] Connection header received
0812 16:32:43.456565 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.456568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.456586 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.456663 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:43.456887 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.458134 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.459995 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.460753 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.460857 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.461000 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.461300 (+ 300us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.461348 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":114,"threads_started":1}
I20260812 16:32:43.461988 29213 tablet_service.cc:1467] Processing CreateTablet for tablet 785830f41fbc4defb5150539772127a6 (DEFAULT_TABLE table=test-table [id=f4bf6a07f034441c825e9c1d621ed4cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:43.462145 29542 tablet_service.cc:1467] Processing CreateTablet for tablet 785830f41fbc4defb5150539772127a6 (DEFAULT_TABLE table=test-table [id=f4bf6a07f034441c825e9c1d621ed4cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:43.462416 29213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 785830f41fbc4defb5150539772127a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:43.462450 29542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 785830f41fbc4defb5150539772127a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:43.465529 29654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.455692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:46241 (local address 127.0.0.1:40562)
0812 16:32:43.459948 (+ 4256us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:43.459966 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:43.460007 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:43.460632 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:43.460633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:43.460635 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:43.460697 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.460699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.463203 (+ 2504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.463204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.464305 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.464309 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.465306 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.465307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.465369 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.465373 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:43.465403 (+ 30us) client_negotiation.cc:240] Negotiation successful
0812 16:32:43.465443 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4243,"thread_start_us":71,"threads_started":1}
I20260812 16:32:43.465925 29656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.455975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40562 (local address 127.19.238.66:46241)
0812 16:32:43.456140 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.456145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.460261 (+ 4116us) server_negotiation.cc:407] Connection header received
0812 16:32:43.460521 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.460524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.460544 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.460629 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:43.460789 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.463072 (+ 2283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.464417 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.465204 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.465665 (+ 461us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.465758 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.465786 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.465828 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260812 16:32:43.466370 29657 tablet_bootstrap.cc:492] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f: Bootstrap starting.
I20260812 16:32:43.466671 29379 tablet_service.cc:1467] Processing CreateTablet for tablet 785830f41fbc4defb5150539772127a6 (DEFAULT_TABLE table=test-table [id=f4bf6a07f034441c825e9c1d621ed4cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:43.467061 29379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 785830f41fbc4defb5150539772127a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:43.467216 29657 tablet_bootstrap.cc:654] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:43.467482 29658 tablet_bootstrap.cc:492] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c: Bootstrap starting.
I20260812 16:32:43.467643 29657 log.cc:824] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:43.468358 29658 tablet_bootstrap.cc:654] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:43.468763 29658 log.cc:824] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:43.469503 29658 tablet_bootstrap.cc:492] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c: No bootstrap required, opened a new log
I20260812 16:32:43.469601 29658 ts_tablet_manager.cc:1397] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 16:32:43.470597 29661 tablet_bootstrap.cc:492] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5: Bootstrap starting.
I20260812 16:32:43.471176 29661 tablet_bootstrap.cc:654] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:43.471278 29657 tablet_bootstrap.cc:492] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f: No bootstrap required, opened a new log
I20260812 16:32:43.471382 29657 ts_tablet_manager.cc:1397] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260812 16:32:43.471405 29661 log.cc:824] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:43.471318 29658 raft_consensus.cc:348] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } } peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } }
I20260812 16:32:43.471503 29658 raft_consensus.cc:374] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:43.471539 29658 raft_consensus.cc:729] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d50f865d8894bb784ce0f6d7654011c, State: Initialized, Role: FOLLOWER
I20260812 16:32:43.471666 29658 consensus_queue.cc:260] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } } peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } }
I20260812 16:32:43.471946 29658 ts_tablet_manager.cc:1428] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260812 16:32:43.472065 29296 heartbeater.cc:503] Master 127.19.238.126:44561 was elected leader, sending a full tablet report...
I20260812 16:32:43.472170 29661 tablet_bootstrap.cc:492] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5: No bootstrap required, opened a new log
I20260812 16:32:43.472231 29661 ts_tablet_manager.cc:1397] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:32:43.473522 29657 raft_consensus.cc:348] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } } peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } }
I20260812 16:32:43.473641 29657 raft_consensus.cc:374] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:43.473665 29657 raft_consensus.cc:729] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8585b614b6cc42c0b9cb217b143ae39f, State: Initialized, Role: FOLLOWER
I20260812 16:32:43.473780 29657 consensus_queue.cc:260] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } } peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } }
I20260812 16:32:43.473871 29661 raft_consensus.cc:348] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } } peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } }
I20260812 16:32:43.474014 29657 ts_tablet_manager.cc:1428] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 16:32:43.474036 29661 raft_consensus.cc:374] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:43.474078 29661 raft_consensus.cc:729] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 976cd6755f3a4a17ba2e220f591ea9f5, State: Initialized, Role: FOLLOWER
I20260812 16:32:43.474115 29624 heartbeater.cc:503] Master 127.19.238.126:44561 was elected leader, sending a full tablet report...
I20260812 16:32:43.474220 29661 consensus_queue.cc:260] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } } peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } }
I20260812 16:32:43.474474 29661 ts_tablet_manager.cc:1428] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 16:32:43.474545 29458 heartbeater.cc:503] Master 127.19.238.126:44561 was elected leader, sending a full tablet report...
W20260812 16:32:43.498950 29459 tablet.cc:2369] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:32:43.536255 29664 raft_consensus.cc:482] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:32:43.536476 29664 raft_consensus.cc:504] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } } peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } }
I20260812 16:32:43.536917 29664 leader_election.cc:288] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d50f865d8894bb784ce0f6d7654011c (127.19.238.65:41743), 976cd6755f3a4a17ba2e220f591ea9f5 (127.19.238.66:46241)
I20260812 16:32:43.540186 29498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.537174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:41743 (local address 127.19.238.67:51763)
0812 16:32:43.537234 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:43.537247 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:43.537295 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:43.537501 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:43.537502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:43.537503 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:43.537601 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.537604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.538335 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.538337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.539187 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.539190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.539815 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.539816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.539945 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.539950 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:32:43.540014 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 16:32:43.540078 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 16:32:43.540315 29656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.537392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:36915 (local address 127.19.238.66:46241)
0812 16:32:43.537449 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.537451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.537579 (+ 128us) server_negotiation.cc:407] Connection header received
0812 16:32:43.537630 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.537631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.537648 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.537703 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:43.537867 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.538439 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.539498 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.539961 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.540024 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.540031 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.540193 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.540238 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 16:32:43.540337 29666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.537307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:46241 (local address 127.19.238.67:36915)
0812 16:32:43.537494 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:43.537506 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:43.537574 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:43.537716 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:43.537717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:43.537719 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:43.537798 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.537800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.538523 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.538523 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.539358 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.539361 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.540046 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.540047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.540112 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.540116 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:43.540163 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 16:32:43.540193 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":132,"threads_started":1}
I20260812 16:32:43.540375 29653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.537297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:51763 (local address 127.19.238.65:41743)
0812 16:32:43.537353 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.537356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.537366 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:43.537399 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.537400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.537410 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.537493 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:43.537704 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.538250 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.539278 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.539737 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.539794 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.539801 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.540259 (+ 458us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.540303 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 16:32:43.540697 29238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "785830f41fbc4defb5150539772127a6" candidate_uuid: "8585b614b6cc42c0b9cb217b143ae39f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d50f865d8894bb784ce0f6d7654011c" is_pre_election: true
I20260812 16:32:43.540709 29402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "785830f41fbc4defb5150539772127a6" candidate_uuid: "8585b614b6cc42c0b9cb217b143ae39f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" is_pre_election: true
I20260812 16:32:43.540884 29402 raft_consensus.cc:2454] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8585b614b6cc42c0b9cb217b143ae39f in term 0.
I20260812 16:32:43.540901 29238 raft_consensus.cc:2454] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8585b614b6cc42c0b9cb217b143ae39f in term 0.
I20260812 16:32:43.541168 29489 leader_election.cc:302] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d50f865d8894bb784ce0f6d7654011c, 8585b614b6cc42c0b9cb217b143ae39f; no voters:
I20260812 16:32:43.541301 29664 raft_consensus.cc:2781] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:32:43.541388 29664 raft_consensus.cc:482] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:32:43.541424 29664 raft_consensus.cc:3037] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:43.542500 29664 raft_consensus.cc:504] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } } peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } }
I20260812 16:32:43.542649 29664 leader_election.cc:288] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [CANDIDATE]: Term 1 election: Requested vote from peers 7d50f865d8894bb784ce0f6d7654011c (127.19.238.65:41743), 976cd6755f3a4a17ba2e220f591ea9f5 (127.19.238.66:46241)
I20260812 16:32:43.542826 29238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "785830f41fbc4defb5150539772127a6" candidate_uuid: "8585b614b6cc42c0b9cb217b143ae39f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d50f865d8894bb784ce0f6d7654011c"
I20260812 16:32:43.542855 29402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "785830f41fbc4defb5150539772127a6" candidate_uuid: "8585b614b6cc42c0b9cb217b143ae39f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "976cd6755f3a4a17ba2e220f591ea9f5"
I20260812 16:32:43.542928 29238 raft_consensus.cc:3037] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:43.542930 29402 raft_consensus.cc:3037] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:43.543658 29402 raft_consensus.cc:2454] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8585b614b6cc42c0b9cb217b143ae39f in term 1.
I20260812 16:32:43.543758 29238 raft_consensus.cc:2454] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8585b614b6cc42c0b9cb217b143ae39f in term 1.
I20260812 16:32:43.543855 29489 leader_election.cc:302] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8585b614b6cc42c0b9cb217b143ae39f, 976cd6755f3a4a17ba2e220f591ea9f5; no voters:
I20260812 16:32:43.543968 29664 raft_consensus.cc:2781] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:43.544124 29664 raft_consensus.cc:686] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [term 1 LEADER]: Becoming Leader. State: Replica: 8585b614b6cc42c0b9cb217b143ae39f, State: Running, Role: LEADER
I20260812 16:32:43.544221 29664 consensus_queue.cc:237] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } } peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } }
I20260812 16:32:43.545073 29053 catalog_manager.cc:5495] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8585b614b6cc42c0b9cb217b143ae39f (127.19.238.67). New cstate: current_term: 1 leader_uuid: "8585b614b6cc42c0b9cb217b143ae39f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } health_report { overall_health: UNKNOWN } } }
I20260812 16:32:43.556730 29655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.554212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38074 (local address 127.19.238.67:35269)
0812 16:32:43.554273 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.554275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.554310 (+ 35us) server_negotiation.cc:407] Connection header received
0812 16:32:43.554387 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.554388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.554403 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.554497 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:43.554617 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.555070 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.555788 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.556057 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.556117 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.556291 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:43.556425 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:43.556523 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.556631 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.556651 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 16:32:43.558713 29402 raft_consensus.cc:1261] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 FOLLOWER]: Refusing update from remote peer 8585b614b6cc42c0b9cb217b143ae39f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:43.558722 29238 raft_consensus.cc:1261] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c [term 1 FOLLOWER]: Refusing update from remote peer 8585b614b6cc42c0b9cb217b143ae39f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:43.559103 29667 consensus_queue.cc:1035] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:43.559254 29664 consensus_queue.cc:1035] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [LEADER]: Connected to new peer: Peer: permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:43.562302 29673 mvcc.cc:205] Tried to move back new op lower bound from 7317718481133334528 to 7317718481077805056. Current Snapshot: MvccSnapshot[applied={T|T < 7317718481133334528}]
I20260812 16:32:43.564603 29052 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:43.564723 29052 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:43.565521 29052 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a: Sending DeleteTablet for 3 replicas of tablet 785830f41fbc4defb5150539772127a6
I20260812 16:32:43.565876 29542 tablet_service.cc:1514] Processing DeleteTablet for tablet 785830f41fbc4defb5150539772127a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:43 UTC) from {username='oryx', principal='oryx/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:38062
I20260812 16:32:43.565900 29213 tablet_service.cc:1514] Processing DeleteTablet for tablet 785830f41fbc4defb5150539772127a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:43 UTC) from {username='oryx', principal='oryx/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:35104
I20260812 16:32:43.566097 29683 tablet_replica.cc:330] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f: stopping tablet replica
I20260812 16:32:43.566143 29379 tablet_service.cc:1514] Processing DeleteTablet for tablet 785830f41fbc4defb5150539772127a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:43 UTC) from {username='oryx', principal='oryx/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:40562
I20260812 16:32:43.566183 29683 raft_consensus.cc:2229] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:32:43.566293 29683 raft_consensus.cc:2258] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:43.566365 29685 tablet_replica.cc:330] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5: stopping tablet replica
I20260812 16:32:43.566464 29685 raft_consensus.cc:2229] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:43.566540 29684 tablet_replica.cc:330] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c: stopping tablet replica
I20260812 16:32:43.566576 29685 raft_consensus.cc:2258] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:43.566689 29684 raft_consensus.cc:2229] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:43.566919 29683 ts_tablet_manager.cc:1905] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:43.566946 29684 raft_consensus.cc:2258] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:43.566974 29685 ts_tablet_manager.cc:1905] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:43.567407 29684 ts_tablet_manager.cc:1905] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:43.568939 29685 ts_tablet_manager.cc:1918] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:32:43.569022 29685 log.cc:1196] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/785830f41fbc4defb5150539772127a6
I20260812 16:32:43.569060 29052 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47098:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:32:43.569235 29052 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:32:43.569319 29685 ts_tablet_manager.cc:1939] T 785830f41fbc4defb5150539772127a6 P 976cd6755f3a4a17ba2e220f591ea9f5: Deleting consensus metadata
I20260812 16:32:43.569448 29684 ts_tablet_manager.cc:1918] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:32:43.569531 29684 log.cc:1196] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/wal/wals/785830f41fbc4defb5150539772127a6
I20260812 16:32:43.569741 29037 catalog_manager.cc:4841] TS 976cd6755f3a4a17ba2e220f591ea9f5 (127.19.238.66:46241): tablet 785830f41fbc4defb5150539772127a6 (table test-table [id=f4bf6a07f034441c825e9c1d621ed4cc]) successfully deleted
I20260812 16:32:43.569788 29684 ts_tablet_manager.cc:1939] T 785830f41fbc4defb5150539772127a6 P 7d50f865d8894bb784ce0f6d7654011c: Deleting consensus metadata
I20260812 16:32:43.570183 29037 catalog_manager.cc:4841] TS 7d50f865d8894bb784ce0f6d7654011c (127.19.238.65:41743): tablet 785830f41fbc4defb5150539772127a6 (table test-table [id=f4bf6a07f034441c825e9c1d621ed4cc]) successfully deleted
I20260812 16:32:43.570628 29683 ts_tablet_manager.cc:1918] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:32:43.570712 29683 log.cc:1196] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/785830f41fbc4defb5150539772127a6
I20260812 16:32:43.570834 29542 tablet_service.cc:1467] Processing CreateTablet for tablet 5218ae85db3a4fbcb85fa85c01a2d115 (DEFAULT_TABLE table=test-table [id=a0ca885e26b04772978d9f1597e8b187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:43.570917 29213 tablet_service.cc:1467] Processing CreateTablet for tablet 5218ae85db3a4fbcb85fa85c01a2d115 (DEFAULT_TABLE table=test-table [id=a0ca885e26b04772978d9f1597e8b187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:43.570950 29542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5218ae85db3a4fbcb85fa85c01a2d115. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:43.571005 29683 ts_tablet_manager.cc:1939] T 785830f41fbc4defb5150539772127a6 P 8585b614b6cc42c0b9cb217b143ae39f: Deleting consensus metadata
I20260812 16:32:43.571026 29213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5218ae85db3a4fbcb85fa85c01a2d115. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:43.571389 29038 catalog_manager.cc:4841] TS 8585b614b6cc42c0b9cb217b143ae39f (127.19.238.67:35269): tablet 785830f41fbc4defb5150539772127a6 (table test-table [id=f4bf6a07f034441c825e9c1d621ed4cc]) successfully deleted
I20260812 16:32:43.571979 29379 tablet_service.cc:1467] Processing CreateTablet for tablet 5218ae85db3a4fbcb85fa85c01a2d115 (DEFAULT_TABLE table=test-table [id=a0ca885e26b04772978d9f1597e8b187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:43.572095 29379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5218ae85db3a4fbcb85fa85c01a2d115. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:43.572584 29657 tablet_bootstrap.cc:492] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f: Bootstrap starting.
I20260812 16:32:43.572625 29658 tablet_bootstrap.cc:492] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c: Bootstrap starting.
I20260812 16:32:43.573019 29658 tablet_bootstrap.cc:654] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:43.573029 29657 tablet_bootstrap.cc:654] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:43.573225 29661 tablet_bootstrap.cc:492] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5: Bootstrap starting.
I20260812 16:32:43.573660 29661 tablet_bootstrap.cc:654] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:43.573769 29658 tablet_bootstrap.cc:492] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c: No bootstrap required, opened a new log
I20260812 16:32:43.573810 29658 ts_tablet_manager.cc:1397] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 16:32:43.573951 29658 raft_consensus.cc:348] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.574018 29658 raft_consensus.cc:374] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:43.574038 29658 raft_consensus.cc:729] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d50f865d8894bb784ce0f6d7654011c, State: Initialized, Role: FOLLOWER
I20260812 16:32:43.574088 29658 consensus_queue.cc:260] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.574182 29658 ts_tablet_manager.cc:1428] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:43.574254 29661 tablet_bootstrap.cc:492] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5: No bootstrap required, opened a new log
I20260812 16:32:43.574309 29661 ts_tablet_manager.cc:1397] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:43.574463 29661 raft_consensus.cc:348] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.574545 29661 raft_consensus.cc:374] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:43.574573 29661 raft_consensus.cc:729] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 976cd6755f3a4a17ba2e220f591ea9f5, State: Initialized, Role: FOLLOWER
I20260812 16:32:43.574622 29661 consensus_queue.cc:260] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.574713 29661 ts_tablet_manager.cc:1428] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:43.576279 29657 tablet_bootstrap.cc:492] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f: No bootstrap required, opened a new log
I20260812 16:32:43.576344 29657 ts_tablet_manager.cc:1397] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20260812 16:32:43.576481 29657 raft_consensus.cc:348] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.576575 29657 raft_consensus.cc:374] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:43.576598 29657 raft_consensus.cc:729] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8585b614b6cc42c0b9cb217b143ae39f, State: Initialized, Role: FOLLOWER
I20260812 16:32:43.576643 29657 consensus_queue.cc:260] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.576740 29657 ts_tablet_manager.cc:1428] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260812 16:32:43.579186 29297 tablet.cc:2369] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:32:43.662892 29625 tablet.cc:2369] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:32:43.673148 29665 raft_consensus.cc:482] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:32:43.673277 29665 raft_consensus.cc:504] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.673682 29665 leader_election.cc:288] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d50f865d8894bb784ce0f6d7654011c (127.19.238.65:41743), 8585b614b6cc42c0b9cb217b143ae39f (127.19.238.67:35269)
I20260812 16:32:43.677304 29655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.674021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:37451 (local address 127.19.238.67:35269)
0812 16:32:43.674075 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.674078 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.674089 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:32:43.674122 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.674123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.674137 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.674206 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:43.674373 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.675081 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.676567 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.676917 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.677011 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.677021 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.677194 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.677235 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 16:32:43.677325 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.673882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:35269 (local address 127.19.238.66:37451)
0812 16:32:43.673952 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:43.673967 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:43.674022 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:43.674220 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:43.674220 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:43.674222 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:43.674297 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.674299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.675155 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.675155 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.676434 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.676437 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.677010 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.677011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.677102 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.677105 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:32:43.677185 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 16:32:43.677229 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":92}
I20260812 16:32:43.677589 29653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.674206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:39517 (local address 127.19.238.65:41743)
0812 16:32:43.674255 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.674256 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.674264 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:32:43.674292 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.674292 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.674302 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.674361 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:43.674537 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.675007 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.676555 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.677163 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.677241 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.677250 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.677465 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.677521 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 16:32:43.677588 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5218ae85db3a4fbcb85fa85c01a2d115" candidate_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8585b614b6cc42c0b9cb217b143ae39f" is_pre_election: true
I20260812 16:32:43.677714 29567 raft_consensus.cc:2454] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 976cd6755f3a4a17ba2e220f591ea9f5 in term 0.
I20260812 16:32:43.677816 29689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.673885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:41743 (local address 127.19.238.66:39517)
0812 16:32:43.674094 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:43.674103 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:43.674147 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:43.674389 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:43.674389 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:43.674391 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:43.674465 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.674466 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.675097 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.675098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.676434 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.676437 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.677303 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.677304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.677375 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.677379 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:43.677683 (+ 304us) client_negotiation.cc:240] Negotiation successful
0812 16:32:43.677726 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":67,"thread_start_us":86,"threads_started":1}
I20260812 16:32:43.677897 29324 leader_election.cc:302] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8585b614b6cc42c0b9cb217b143ae39f, 976cd6755f3a4a17ba2e220f591ea9f5; no voters:
I20260812 16:32:43.678052 29665 raft_consensus.cc:2781] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:32:43.678088 29665 raft_consensus.cc:482] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:32:43.678061 29238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5218ae85db3a4fbcb85fa85c01a2d115" candidate_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d50f865d8894bb784ce0f6d7654011c" is_pre_election: true
I20260812 16:32:43.678117 29665 raft_consensus.cc:3037] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:43.678145 29238 raft_consensus.cc:2454] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 976cd6755f3a4a17ba2e220f591ea9f5 in term 0.
I20260812 16:32:43.678900 29665 raft_consensus.cc:504] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.679095 29665 leader_election.cc:288] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [CANDIDATE]: Term 1 election: Requested vote from peers 7d50f865d8894bb784ce0f6d7654011c (127.19.238.65:41743), 8585b614b6cc42c0b9cb217b143ae39f (127.19.238.67:35269)
I20260812 16:32:43.679283 29238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5218ae85db3a4fbcb85fa85c01a2d115" candidate_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d50f865d8894bb784ce0f6d7654011c"
I20260812 16:32:43.679360 29238 raft_consensus.cc:3037] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:43.679292 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5218ae85db3a4fbcb85fa85c01a2d115" candidate_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8585b614b6cc42c0b9cb217b143ae39f"
I20260812 16:32:43.679392 29567 raft_consensus.cc:3037] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:43.680137 29567 raft_consensus.cc:2454] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 976cd6755f3a4a17ba2e220f591ea9f5 in term 1.
I20260812 16:32:43.680156 29238 raft_consensus.cc:2454] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 976cd6755f3a4a17ba2e220f591ea9f5 in term 1.
I20260812 16:32:43.680325 29324 leader_election.cc:302] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8585b614b6cc42c0b9cb217b143ae39f, 976cd6755f3a4a17ba2e220f591ea9f5; no voters:
I20260812 16:32:43.680446 29665 raft_consensus.cc:2781] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:43.680631 29665 raft_consensus.cc:686] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 LEADER]: Becoming Leader. State: Replica: 976cd6755f3a4a17ba2e220f591ea9f5, State: Running, Role: LEADER
I20260812 16:32:43.680711 29665 consensus_queue.cc:237] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.681627 29053 catalog_manager.cc:5495] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 976cd6755f3a4a17ba2e220f591ea9f5 (127.19.238.66). New cstate: current_term: 1 leader_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } health_report { overall_health: UNKNOWN } } }
Aug 12 16:32:43 dist-test-slave-mvvj krb5kdc[29001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552362, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.126@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM
I20260812 16:32:43.724323 29697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.716693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:44561 (local address 127.0.0.1:47100)
0812 16:32:43.716911 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:43.716921 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:43.717008 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:43.717136 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:43.717136 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:43.717472 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:43.717534 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.717536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.718273 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.718273 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.718861 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.718863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.719234 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.719236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.719374 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.720072 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:43.720077 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:43.721588 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:43.723178 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:43.723185 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:43.723187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:43.723434 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:43.723656 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:43.723657 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:43.723657 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:43.723724 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:43.724016 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:43.724022 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:43.724066 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 16:32:43.724127 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 16:32:43.724226 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":142,"threads_started":1}
I20260812 16:32:43.724375 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.716764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47100 (local address 127.19.238.126:44561)
0812 16:32:43.716882 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.716888 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.717011 (+ 123us) server_negotiation.cc:407] Connection header received
0812 16:32:43.717054 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.717055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.717066 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.717123 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:43.717651 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.718176 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.718943 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.719137 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.719199 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.721685 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:43.721686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:43.721686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:43.721693 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:43.723102 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:43.723504 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:43.723504 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:43.723505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:43.723575 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:43.723789 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:43.723790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:43.723790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:43.723953 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:43.724006 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.724159 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.724275 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":76}
I20260812 16:32:43.726486 29053 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:47100:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 16:32:43.726716 29053 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:32:43.728415 29542 tablet_service.cc:1467] Processing CreateTablet for tablet fd041abf040441bbaab40415eb3ed10a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d32a4283304a4ae7b45299ba7bb293ba]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:32:43.728431 29213 tablet_service.cc:1467] Processing CreateTablet for tablet fd041abf040441bbaab40415eb3ed10a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d32a4283304a4ae7b45299ba7bb293ba]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:32:43.728629 29213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd041abf040441bbaab40415eb3ed10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:43.728638 29542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd041abf040441bbaab40415eb3ed10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:43.729815 29658 tablet_bootstrap.cc:492] T fd041abf040441bbaab40415eb3ed10a P 7d50f865d8894bb784ce0f6d7654011c: Bootstrap starting.
I20260812 16:32:43.729961 29379 tablet_service.cc:1467] Processing CreateTablet for tablet fd041abf040441bbaab40415eb3ed10a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d32a4283304a4ae7b45299ba7bb293ba]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:32:43.730121 29379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd041abf040441bbaab40415eb3ed10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:43.730252 29658 tablet_bootstrap.cc:654] T fd041abf040441bbaab40415eb3ed10a P 7d50f865d8894bb784ce0f6d7654011c: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:43.730917 29658 tablet_bootstrap.cc:492] T fd041abf040441bbaab40415eb3ed10a P 7d50f865d8894bb784ce0f6d7654011c: No bootstrap required, opened a new log
I20260812 16:32:43.730962 29658 ts_tablet_manager.cc:1397] T fd041abf040441bbaab40415eb3ed10a P 7d50f865d8894bb784ce0f6d7654011c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 16:32:43.731105 29658 raft_consensus.cc:348] T fd041abf040441bbaab40415eb3ed10a P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.731180 29658 raft_consensus.cc:374] T fd041abf040441bbaab40415eb3ed10a P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:43.731194 29658 raft_consensus.cc:729] T fd041abf040441bbaab40415eb3ed10a P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d50f865d8894bb784ce0f6d7654011c, State: Initialized, Role: FOLLOWER
I20260812 16:32:43.731254 29658 consensus_queue.cc:260] T fd041abf040441bbaab40415eb3ed10a P 7d50f865d8894bb784ce0f6d7654011c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.731362 29658 ts_tablet_manager.cc:1428] T fd041abf040441bbaab40415eb3ed10a P 7d50f865d8894bb784ce0f6d7654011c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:43.731451 29657 tablet_bootstrap.cc:492] T fd041abf040441bbaab40415eb3ed10a P 8585b614b6cc42c0b9cb217b143ae39f: Bootstrap starting.
I20260812 16:32:43.731410 29663 tablet_replica.cc:439] T fd041abf040441bbaab40415eb3ed10a P 7d50f865d8894bb784ce0f6d7654011c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } } }
I20260812 16:32:43.731503 29663 tablet_replica.cc:442] T fd041abf040441bbaab40415eb3ed10a P 7d50f865d8894bb784ce0f6d7654011c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:43.731805 29657 tablet_bootstrap.cc:654] T fd041abf040441bbaab40415eb3ed10a P 8585b614b6cc42c0b9cb217b143ae39f: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:43.732128 29661 tablet_bootstrap.cc:492] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5: Bootstrap starting.
I20260812 16:32:43.732473 29657 tablet_bootstrap.cc:492] T fd041abf040441bbaab40415eb3ed10a P 8585b614b6cc42c0b9cb217b143ae39f: No bootstrap required, opened a new log
I20260812 16:32:43.732534 29657 ts_tablet_manager.cc:1397] T fd041abf040441bbaab40415eb3ed10a P 8585b614b6cc42c0b9cb217b143ae39f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:43.732553 29661 tablet_bootstrap.cc:654] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:43.732647 29657 raft_consensus.cc:348] T fd041abf040441bbaab40415eb3ed10a P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.732731 29657 raft_consensus.cc:374] T fd041abf040441bbaab40415eb3ed10a P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:43.732758 29657 raft_consensus.cc:729] T fd041abf040441bbaab40415eb3ed10a P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8585b614b6cc42c0b9cb217b143ae39f, State: Initialized, Role: FOLLOWER
I20260812 16:32:43.732812 29657 consensus_queue.cc:260] T fd041abf040441bbaab40415eb3ed10a P 8585b614b6cc42c0b9cb217b143ae39f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.732919 29657 ts_tablet_manager.cc:1428] T fd041abf040441bbaab40415eb3ed10a P 8585b614b6cc42c0b9cb217b143ae39f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:43.732954 29679 tablet_replica.cc:439] T fd041abf040441bbaab40415eb3ed10a P 8585b614b6cc42c0b9cb217b143ae39f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } } }
I20260812 16:32:43.733033 29679 tablet_replica.cc:442] T fd041abf040441bbaab40415eb3ed10a P 8585b614b6cc42c0b9cb217b143ae39f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:43.733243 29661 tablet_bootstrap.cc:492] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5: No bootstrap required, opened a new log
I20260812 16:32:43.733297 29661 ts_tablet_manager.cc:1397] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:43.733445 29661 raft_consensus.cc:348] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.733520 29661 raft_consensus.cc:374] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:43.733544 29661 raft_consensus.cc:729] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 976cd6755f3a4a17ba2e220f591ea9f5, State: Initialized, Role: FOLLOWER
I20260812 16:32:43.733596 29661 consensus_queue.cc:260] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.733691 29661 ts_tablet_manager.cc:1428] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:43.733784 29690 tablet_replica.cc:439] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } } }
I20260812 16:32:43.733887 29690 tablet_replica.cc:442] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:43.875976 29690 raft_consensus.cc:482] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:32:43.876097 29690 raft_consensus.cc:504] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.876298 29690 leader_election.cc:288] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d50f865d8894bb784ce0f6d7654011c (127.19.238.65:41743), 8585b614b6cc42c0b9cb217b143ae39f (127.19.238.67:35269)
I20260812 16:32:43.876634 29238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd041abf040441bbaab40415eb3ed10a" candidate_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d50f865d8894bb784ce0f6d7654011c" is_pre_election: true
I20260812 16:32:43.876751 29238 raft_consensus.cc:2454] T fd041abf040441bbaab40415eb3ed10a P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 976cd6755f3a4a17ba2e220f591ea9f5 in term 0.
I20260812 16:32:43.876688 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd041abf040441bbaab40415eb3ed10a" candidate_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8585b614b6cc42c0b9cb217b143ae39f" is_pre_election: true
I20260812 16:32:43.876803 29567 raft_consensus.cc:2454] T fd041abf040441bbaab40415eb3ed10a P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 976cd6755f3a4a17ba2e220f591ea9f5 in term 0.
I20260812 16:32:43.876994 29323 leader_election.cc:302] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d50f865d8894bb784ce0f6d7654011c, 976cd6755f3a4a17ba2e220f591ea9f5; no voters:
I20260812 16:32:43.877125 29690 raft_consensus.cc:2781] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:32:43.877187 29690 raft_consensus.cc:482] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:32:43.877226 29690 raft_consensus.cc:3037] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:43.878011 29690 raft_consensus.cc:504] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.878175 29690 leader_election.cc:288] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 [CANDIDATE]: Term 1 election: Requested vote from peers 7d50f865d8894bb784ce0f6d7654011c (127.19.238.65:41743), 8585b614b6cc42c0b9cb217b143ae39f (127.19.238.67:35269)
I20260812 16:32:43.878394 29238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd041abf040441bbaab40415eb3ed10a" candidate_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d50f865d8894bb784ce0f6d7654011c"
I20260812 16:32:43.878477 29238 raft_consensus.cc:3037] T fd041abf040441bbaab40415eb3ed10a P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:43.878423 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd041abf040441bbaab40415eb3ed10a" candidate_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8585b614b6cc42c0b9cb217b143ae39f"
I20260812 16:32:43.878526 29567 raft_consensus.cc:3037] T fd041abf040441bbaab40415eb3ed10a P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:43.879141 29238 raft_consensus.cc:2454] T fd041abf040441bbaab40415eb3ed10a P 7d50f865d8894bb784ce0f6d7654011c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 976cd6755f3a4a17ba2e220f591ea9f5 in term 1.
I20260812 16:32:43.879316 29323 leader_election.cc:302] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d50f865d8894bb784ce0f6d7654011c, 976cd6755f3a4a17ba2e220f591ea9f5; no voters:
I20260812 16:32:43.879321 29567 raft_consensus.cc:2454] T fd041abf040441bbaab40415eb3ed10a P 8585b614b6cc42c0b9cb217b143ae39f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 976cd6755f3a4a17ba2e220f591ea9f5 in term 1.
I20260812 16:32:43.879420 29690 raft_consensus.cc:2781] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:43.879479 29690 raft_consensus.cc:686] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 LEADER]: Becoming Leader. State: Replica: 976cd6755f3a4a17ba2e220f591ea9f5, State: Running, Role: LEADER
I20260812 16:32:43.879531 29690 consensus_queue.cc:237] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.879897 29665 tablet_replica.cc:439] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5: TxnStatusTablet state changed. Reason: New leader 976cd6755f3a4a17ba2e220f591ea9f5. Latest consensus state: current_term: 1 leader_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } } }
I20260812 16:32:43.880010 29665 tablet_replica.cc:442] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:32:43.880244 29052 catalog_manager.cc:5495] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 976cd6755f3a4a17ba2e220f591ea9f5 (127.19.238.66). New cstate: current_term: 1 leader_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } health_report { overall_health: UNKNOWN } } }
I20260812 16:32:43.880597 29702 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 16:32:43.880746 29702 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 16:32:43.915143 29656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.912415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40576 (local address 127.19.238.66:46241)
0812 16:32:43.912472 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.912474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.912484 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:43.912621 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.912623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.912645 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.912729 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:43.912878 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.913412 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.914209 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.914451 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.914544 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.914792 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:43.914899 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:43.914958 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.915057 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.915078 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 16:32:43.915194 29697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.912342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:46241 (local address 127.0.0.1:40576)
0812 16:32:43.912431 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:43.912446 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:43.912508 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:43.912740 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:43.912741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:43.912742 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:32:43.912799 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.912801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.913512 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.913513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.914103 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.914105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.914569 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.914571 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.914657 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.914679 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:32:43.914969 (+ 290us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:32:43.914979 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 16:32:43.915039 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 16:32:43.915099 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 16:32:43.916558 29238 raft_consensus.cc:1261] T fd041abf040441bbaab40415eb3ed10a P 7d50f865d8894bb784ce0f6d7654011c [term 1 FOLLOWER]: Refusing update from remote peer 976cd6755f3a4a17ba2e220f591ea9f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:43.916635 29567 raft_consensus.cc:1261] T fd041abf040441bbaab40415eb3ed10a P 8585b614b6cc42c0b9cb217b143ae39f [term 1 FOLLOWER]: Refusing update from remote peer 976cd6755f3a4a17ba2e220f591ea9f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:43.916883 29665 consensus_queue.cc:1035] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:43.917013 29690 consensus_queue.cc:1035] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:43.917577 29663 tablet_replica.cc:439] T fd041abf040441bbaab40415eb3ed10a P 7d50f865d8894bb784ce0f6d7654011c: TxnStatusTablet state changed. Reason: New leader 976cd6755f3a4a17ba2e220f591ea9f5. Latest consensus state: current_term: 1 leader_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } } }
I20260812 16:32:43.917680 29663 tablet_replica.cc:442] T fd041abf040441bbaab40415eb3ed10a P 7d50f865d8894bb784ce0f6d7654011c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:43.917768 29679 tablet_replica.cc:439] T fd041abf040441bbaab40415eb3ed10a P 8585b614b6cc42c0b9cb217b143ae39f: TxnStatusTablet state changed. Reason: New leader 976cd6755f3a4a17ba2e220f591ea9f5. Latest consensus state: current_term: 1 leader_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } } }
I20260812 16:32:43.917873 29679 tablet_replica.cc:442] T fd041abf040441bbaab40415eb3ed10a P 8585b614b6cc42c0b9cb217b143ae39f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:43.918691 29690 tablet_replica.cc:439] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } } }
I20260812 16:32:43.918787 29690 tablet_replica.cc:442] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:32:43.918843 29679 tablet_replica.cc:439] T fd041abf040441bbaab40415eb3ed10a P 8585b614b6cc42c0b9cb217b143ae39f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } } }
I20260812 16:32:43.918880 29690 tablet_replica.cc:439] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } } }
I20260812 16:32:43.918938 29679 tablet_replica.cc:442] T fd041abf040441bbaab40415eb3ed10a P 8585b614b6cc42c0b9cb217b143ae39f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:43.918946 29690 tablet_replica.cc:442] T fd041abf040441bbaab40415eb3ed10a P 976cd6755f3a4a17ba2e220f591ea9f5: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:32:43.919369 29663 tablet_replica.cc:439] T fd041abf040441bbaab40415eb3ed10a P 7d50f865d8894bb784ce0f6d7654011c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } } }
I20260812 16:32:43.919454 29663 tablet_replica.cc:442] T fd041abf040441bbaab40415eb3ed10a P 7d50f865d8894bb784ce0f6d7654011c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:32:43.919610 29672 mvcc.cc:205] Tried to move back new op lower bound from 7317718482599804928 to 7317718482451148800. Current Snapshot: MvccSnapshot[applied={T|T < 7317718482599804928 or (T in {7317718482599804928})}]
I20260812 16:32:43.923954 29656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.920961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40592 (local address 127.19.238.66:46241)
0812 16:32:43.921000 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.921002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.921034 (+ 32us) server_negotiation.cc:407] Connection header received
0812 16:32:43.921109 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.921109 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.921119 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.921166 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:43.921281 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.921873 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.923018 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.923287 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.923415 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.923607 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:43.923641 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:43.923703 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.923776 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.923799 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:32:43.929804 29656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.927025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:55897 (local address 127.19.238.66:46241)
0812 16:32:43.927073 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.927075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.927223 (+ 148us) server_negotiation.cc:407] Connection header received
0812 16:32:43.927267 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.927268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.927278 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.927346 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:43.927473 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.928221 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.929033 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.929325 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.929398 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.929563 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:43.929593 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:43.929642 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.929691 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.929709 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 16:32:43.929816 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.926968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:46241 (local address 127.19.238.66:55897)
0812 16:32:43.927149 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:43.927161 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:43.927224 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:43.927347 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:43.927348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:43.927358 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:32:43.927415 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.927416 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.928303 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.928304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.928949 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:43.928952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.929404 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.929405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:43.929464 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.929481 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:32:43.929653 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:32:43.929654 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 16:32:43.929689 (+ 35us) client_negotiation.cc:240] Negotiation successful
0812 16:32:43.929723 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":111,"threads_started":1}
I20260812 16:32:43.932551 29238 raft_consensus.cc:1261] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c [term 1 FOLLOWER]: Refusing update from remote peer 976cd6755f3a4a17ba2e220f591ea9f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:43.932551 29567 raft_consensus.cc:1261] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f [term 1 FOLLOWER]: Refusing update from remote peer 976cd6755f3a4a17ba2e220f591ea9f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:43.932873 29711 consensus_queue.cc:1035] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:43.933044 29690 consensus_queue.cc:1035] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:43.945892 29053 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:43.946043 29053 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:43.946758 29053 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a: Sending DeleteTablet for 3 replicas of tablet 5218ae85db3a4fbcb85fa85c01a2d115
I20260812 16:32:43.947108 29379 tablet_service.cc:1514] Processing DeleteTablet for tablet 5218ae85db3a4fbcb85fa85c01a2d115 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:43 UTC) from {username='oryx', principal='oryx/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:40562
I20260812 16:32:43.947117 29213 tablet_service.cc:1514] Processing DeleteTablet for tablet 5218ae85db3a4fbcb85fa85c01a2d115 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:43 UTC) from {username='oryx', principal='oryx/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:35104
I20260812 16:32:43.947155 29542 tablet_service.cc:1514] Processing DeleteTablet for tablet 5218ae85db3a4fbcb85fa85c01a2d115 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:43 UTC) from {username='oryx', principal='oryx/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:38062
I20260812 16:32:43.947248 29684 tablet_replica.cc:330] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c: stopping tablet replica
I20260812 16:32:43.947252 29685 tablet_replica.cc:330] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5: stopping tablet replica
I20260812 16:32:43.947260 29683 tablet_replica.cc:330] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f: stopping tablet replica
I20260812 16:32:43.947314 29685 raft_consensus.cc:2229] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:32:43.947317 29684 raft_consensus.cc:2229] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:43.947325 29683 raft_consensus.cc:2229] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:43.947365 29684 raft_consensus.cc:2258] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:43.947368 29683 raft_consensus.cc:2258] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:43.947417 29685 raft_consensus.cc:2258] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:43.947760 29684 ts_tablet_manager.cc:1905] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:43.947960 29685 ts_tablet_manager.cc:1905] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:43.949342 29684 ts_tablet_manager.cc:1918] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:32:43.949424 29684 log.cc:1196] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/wal/wals/5218ae85db3a4fbcb85fa85c01a2d115
I20260812 16:32:43.949587 29685 ts_tablet_manager.cc:1918] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:32:43.949654 29684 ts_tablet_manager.cc:1939] T 5218ae85db3a4fbcb85fa85c01a2d115 P 7d50f865d8894bb784ce0f6d7654011c: Deleting consensus metadata
I20260812 16:32:43.949671 29685 log.cc:1196] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/5218ae85db3a4fbcb85fa85c01a2d115
I20260812 16:32:43.949899 29685 ts_tablet_manager.cc:1939] T 5218ae85db3a4fbcb85fa85c01a2d115 P 976cd6755f3a4a17ba2e220f591ea9f5: Deleting consensus metadata
I20260812 16:32:43.950067 29037 catalog_manager.cc:4841] TS 7d50f865d8894bb784ce0f6d7654011c (127.19.238.65:41743): tablet 5218ae85db3a4fbcb85fa85c01a2d115 (table test-table [id=a0ca885e26b04772978d9f1597e8b187]) successfully deleted
I20260812 16:32:43.950176 29037 catalog_manager.cc:4841] TS 976cd6755f3a4a17ba2e220f591ea9f5 (127.19.238.66:46241): tablet 5218ae85db3a4fbcb85fa85c01a2d115 (table test-table [id=a0ca885e26b04772978d9f1597e8b187]) successfully deleted
I20260812 16:32:43.950534 29683 ts_tablet_manager.cc:1905] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:43.952261 29683 ts_tablet_manager.cc:1918] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:32:43.952342 29683 log.cc:1196] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/5218ae85db3a4fbcb85fa85c01a2d115
I20260812 16:32:43.952579 29683 ts_tablet_manager.cc:1939] T 5218ae85db3a4fbcb85fa85c01a2d115 P 8585b614b6cc42c0b9cb217b143ae39f: Deleting consensus metadata
I20260812 16:32:43.952963 29038 catalog_manager.cc:4841] TS 8585b614b6cc42c0b9cb217b143ae39f (127.19.238.67:35269): tablet 5218ae85db3a4fbcb85fa85c01a2d115 (table test-table [id=a0ca885e26b04772978d9f1597e8b187]) successfully deleted
Aug 12 16:32:43 dist-test-slave-mvvj krb5kdc[29001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 16:32:43 dist-test-slave-mvvj krb5kdc[29001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:43 dist-test-slave-mvvj krb5kdc[29001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 16:32:43.976327 29112 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:43.970178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47104 (local address 127.19.238.126:44561)
0812 16:32:43.970242 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.970245 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.970359 (+ 114us) server_negotiation.cc:407] Connection header received
0812 16:32:43.970446 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.970486 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.970500 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.970572 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:43.971245 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.971758 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.972589 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.972790 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.972871 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.976243 (+ 3372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47104: BlockingRecv error: recv got EOF from 127.0.0.1:47104 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 12 16:32:43 dist-test-slave-mvvj krb5kdc[29001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM
I20260812 16:32:43.985167 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:43.977372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47118 (local address 127.19.238.126:44561)
0812 16:32:43.977425 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:43.977427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:43.977511 (+ 84us) server_negotiation.cc:407] Connection header received
0812 16:32:43.977567 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:43.977568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:43.977581 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:43.977649 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:43.978098 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.978861 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.979604 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:43.979939 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:43.980021 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:43.982357 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:43.982359 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:43.982359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:43.982368 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:43.984015 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:43.984367 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:43.984368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:43.984368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:43.984439 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:43.984638 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:43.984640 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:43.984640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:43.984791 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:43.984909 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:43.984980 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 16:32:43.985067 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 16:32:43.987063 29053 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47118:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:32:43.987295 29053 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:32:43.988943 29542 tablet_service.cc:1467] Processing CreateTablet for tablet deedfcfd58d240edb019fac2777cdd90 (DEFAULT_TABLE table=test-table [id=baf7abf5cb864a5286d122ceeb08eed3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:43.988953 29213 tablet_service.cc:1467] Processing CreateTablet for tablet deedfcfd58d240edb019fac2777cdd90 (DEFAULT_TABLE table=test-table [id=baf7abf5cb864a5286d122ceeb08eed3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:43.989123 29542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deedfcfd58d240edb019fac2777cdd90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:43.989125 29213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deedfcfd58d240edb019fac2777cdd90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:43.990334 29657 tablet_bootstrap.cc:492] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f: Bootstrap starting.
I20260812 16:32:43.990439 29379 tablet_service.cc:1467] Processing CreateTablet for tablet deedfcfd58d240edb019fac2777cdd90 (DEFAULT_TABLE table=test-table [id=baf7abf5cb864a5286d122ceeb08eed3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:43.990589 29379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deedfcfd58d240edb019fac2777cdd90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:43.990621 29658 tablet_bootstrap.cc:492] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c: Bootstrap starting.
I20260812 16:32:43.990769 29657 tablet_bootstrap.cc:654] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:43.991070 29658 tablet_bootstrap.cc:654] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:43.991799 29658 tablet_bootstrap.cc:492] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c: No bootstrap required, opened a new log
I20260812 16:32:43.991986 29658 ts_tablet_manager.cc:1397] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:43.992048 29661 tablet_bootstrap.cc:492] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5: Bootstrap starting.
I20260812 16:32:43.992209 29658 raft_consensus.cc:348] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.992306 29658 raft_consensus.cc:374] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:43.992341 29658 raft_consensus.cc:729] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d50f865d8894bb784ce0f6d7654011c, State: Initialized, Role: FOLLOWER
I20260812 16:32:43.992404 29658 consensus_queue.cc:260] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.992514 29661 tablet_bootstrap.cc:654] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:43.992525 29658 ts_tablet_manager.cc:1428] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 16:32:43.993182 29661 tablet_bootstrap.cc:492] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5: No bootstrap required, opened a new log
I20260812 16:32:43.993237 29661 ts_tablet_manager.cc:1397] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:43.993243 29657 tablet_bootstrap.cc:492] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f: No bootstrap required, opened a new log
I20260812 16:32:43.993288 29657 ts_tablet_manager.cc:1397] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 16:32:43.993407 29657 raft_consensus.cc:348] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.993407 29661 raft_consensus.cc:348] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.993468 29661 raft_consensus.cc:374] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:43.993468 29657 raft_consensus.cc:374] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:43.993490 29661 raft_consensus.cc:729] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 976cd6755f3a4a17ba2e220f591ea9f5, State: Initialized, Role: FOLLOWER
I20260812 16:32:43.993490 29657 raft_consensus.cc:729] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8585b614b6cc42c0b9cb217b143ae39f, State: Initialized, Role: FOLLOWER
I20260812 16:32:43.993546 29661 consensus_queue.cc:260] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.993553 29657 consensus_queue.cc:260] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:43.993636 29661 ts_tablet_manager.cc:1428] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:43.993642 29657 ts_tablet_manager.cc:1428] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:44.142203 29679 raft_consensus.cc:482] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:32:44.142342 29679 raft_consensus.cc:504] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:44.142601 29679 leader_election.cc:288] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d50f865d8894bb784ce0f6d7654011c (127.19.238.65:41743), 976cd6755f3a4a17ba2e220f591ea9f5 (127.19.238.66:46241)
I20260812 16:32:44.142859 29238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deedfcfd58d240edb019fac2777cdd90" candidate_uuid: "8585b614b6cc42c0b9cb217b143ae39f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d50f865d8894bb784ce0f6d7654011c" is_pre_election: true
I20260812 16:32:44.142972 29238 raft_consensus.cc:2454] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8585b614b6cc42c0b9cb217b143ae39f in term 0.
I20260812 16:32:44.143122 29402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deedfcfd58d240edb019fac2777cdd90" candidate_uuid: "8585b614b6cc42c0b9cb217b143ae39f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" is_pre_election: true
I20260812 16:32:44.143234 29402 raft_consensus.cc:2454] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8585b614b6cc42c0b9cb217b143ae39f in term 0.
I20260812 16:32:44.143687 29489 leader_election.cc:302] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8585b614b6cc42c0b9cb217b143ae39f, 976cd6755f3a4a17ba2e220f591ea9f5; no voters:
I20260812 16:32:44.143827 29679 raft_consensus.cc:2781] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:32:44.143889 29679 raft_consensus.cc:482] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:32:44.143910 29679 raft_consensus.cc:3037] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:44.145056 29679 raft_consensus.cc:504] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:44.145203 29679 leader_election.cc:288] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [CANDIDATE]: Term 1 election: Requested vote from peers 7d50f865d8894bb784ce0f6d7654011c (127.19.238.65:41743), 976cd6755f3a4a17ba2e220f591ea9f5 (127.19.238.66:46241)
I20260812 16:32:44.145581 29238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deedfcfd58d240edb019fac2777cdd90" candidate_uuid: "8585b614b6cc42c0b9cb217b143ae39f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d50f865d8894bb784ce0f6d7654011c"
I20260812 16:32:44.145658 29402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deedfcfd58d240edb019fac2777cdd90" candidate_uuid: "8585b614b6cc42c0b9cb217b143ae39f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "976cd6755f3a4a17ba2e220f591ea9f5"
I20260812 16:32:44.145740 29402 raft_consensus.cc:3037] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:44.145720 29238 raft_consensus.cc:3037] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:44.146633 29402 raft_consensus.cc:2454] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8585b614b6cc42c0b9cb217b143ae39f in term 1.
I20260812 16:32:44.146668 29238 raft_consensus.cc:2454] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8585b614b6cc42c0b9cb217b143ae39f in term 1.
I20260812 16:32:44.146835 29489 leader_election.cc:302] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d50f865d8894bb784ce0f6d7654011c, 8585b614b6cc42c0b9cb217b143ae39f; no voters:
I20260812 16:32:44.146952 29679 raft_consensus.cc:2781] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:44.147094 29679 raft_consensus.cc:686] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [term 1 LEADER]: Becoming Leader. State: Replica: 8585b614b6cc42c0b9cb217b143ae39f, State: Running, Role: LEADER
I20260812 16:32:44.147158 29679 consensus_queue.cc:237] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:44.147907 29052 catalog_manager.cc:5495] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8585b614b6cc42c0b9cb217b143ae39f (127.19.238.67). New cstate: current_term: 1 leader_uuid: "8585b614b6cc42c0b9cb217b143ae39f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } health_report { overall_health: HEALTHY } } }
I20260812 16:32:44.178723 29655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:44.176185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38084 (local address 127.19.238.67:35269)
0812 16:32:44.176231 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:44.176233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:44.176241 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:32:44.176305 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:44.176305 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:44.176314 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:44.176364 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:44.176593 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.177055 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.177907 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.178168 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.178250 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:44.178428 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:44.178477 (+ 49us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:44.178542 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:44.178609 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 16:32:44.178634 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 16:32:44.179939 29402 raft_consensus.cc:1261] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 FOLLOWER]: Refusing update from remote peer 8585b614b6cc42c0b9cb217b143ae39f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:44.179939 29238 raft_consensus.cc:1261] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c [term 1 FOLLOWER]: Refusing update from remote peer 8585b614b6cc42c0b9cb217b143ae39f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:44.180186 29731 consensus_queue.cc:1035] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [LEADER]: Connected to new peer: Peer: permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:44.180326 29679 consensus_queue.cc:1035] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:44.181943 29671 mvcc.cc:205] Tried to move back new op lower bound from 7317718483678556160 to 7317718483547291648. Current Snapshot: MvccSnapshot[applied={T|T < 7317718483678556160 or (T in {7317718483678556160})}]
I20260812 16:32:44.183152 29052 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:44.183259 29052 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:44.183920 29052 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a: Sending DeleteTablet for 3 replicas of tablet deedfcfd58d240edb019fac2777cdd90
I20260812 16:32:44.184203 29542 tablet_service.cc:1514] Processing DeleteTablet for tablet deedfcfd58d240edb019fac2777cdd90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:44 UTC) from {username='oryx', principal='oryx/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:38062
I20260812 16:32:44.184333 29379 tablet_service.cc:1514] Processing DeleteTablet for tablet deedfcfd58d240edb019fac2777cdd90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:44 UTC) from {username='oryx', principal='oryx/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:40562
I20260812 16:32:44.184499 29685 tablet_replica.cc:330] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5: stopping tablet replica
I20260812 16:32:44.184419 29052 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47118:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 16:32:44.184507 29213 tablet_service.cc:1514] Processing DeleteTablet for tablet deedfcfd58d240edb019fac2777cdd90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:44 UTC) from {username='oryx', principal='oryx/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:35104
I20260812 16:32:44.184558 29685 raft_consensus.cc:2229] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:44.184605 29685 raft_consensus.cc:2258] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:44.184602 29683 tablet_replica.cc:330] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f: stopping tablet replica
W20260812 16:32:44.184609 29052 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:32:44.184621 29684 tablet_replica.cc:330] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c: stopping tablet replica
I20260812 16:32:44.184659 29683 raft_consensus.cc:2229] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:32:44.184679 29684 raft_consensus.cc:2229] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:44.184713 29683 raft_consensus.cc:2258] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:44.184726 29684 raft_consensus.cc:2258] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:44.185041 29683 ts_tablet_manager.cc:1905] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:44.185046 29684 ts_tablet_manager.cc:1905] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:44.185119 29685 ts_tablet_manager.cc:1905] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:44.186638 29683 ts_tablet_manager.cc:1918] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:32:44.186643 29685 ts_tablet_manager.cc:1918] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:32:44.186697 29685 log.cc:1196] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/deedfcfd58d240edb019fac2777cdd90
I20260812 16:32:44.186697 29683 log.cc:1196] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/deedfcfd58d240edb019fac2777cdd90
I20260812 16:32:44.186683 29684 ts_tablet_manager.cc:1918] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:32:44.186741 29684 log.cc:1196] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-0/wal/wals/deedfcfd58d240edb019fac2777cdd90
I20260812 16:32:44.186932 29683 ts_tablet_manager.cc:1939] T deedfcfd58d240edb019fac2777cdd90 P 8585b614b6cc42c0b9cb217b143ae39f: Deleting consensus metadata
I20260812 16:32:44.186937 29685 ts_tablet_manager.cc:1939] T deedfcfd58d240edb019fac2777cdd90 P 976cd6755f3a4a17ba2e220f591ea9f5: Deleting consensus metadata
I20260812 16:32:44.186985 29684 ts_tablet_manager.cc:1939] T deedfcfd58d240edb019fac2777cdd90 P 7d50f865d8894bb784ce0f6d7654011c: Deleting consensus metadata
I20260812 16:32:44.187307 29213 tablet_service.cc:1467] Processing CreateTablet for tablet 45cc41f0cb8146dca880e4c08d271794 (DEFAULT_TABLE table=test-table [id=e5f90405883441318a549dfbb2dcb5ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:44.187350 29038 catalog_manager.cc:4841] TS 8585b614b6cc42c0b9cb217b143ae39f (127.19.238.67:35269): tablet deedfcfd58d240edb019fac2777cdd90 (table test-table [id=baf7abf5cb864a5286d122ceeb08eed3]) successfully deleted
I20260812 16:32:44.187433 29213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45cc41f0cb8146dca880e4c08d271794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:44.187500 29037 catalog_manager.cc:4841] TS 976cd6755f3a4a17ba2e220f591ea9f5 (127.19.238.66:46241): tablet deedfcfd58d240edb019fac2777cdd90 (table test-table [id=baf7abf5cb864a5286d122ceeb08eed3]) successfully deleted
I20260812 16:32:44.187938 29542 tablet_service.cc:1467] Processing CreateTablet for tablet 45cc41f0cb8146dca880e4c08d271794 (DEFAULT_TABLE table=test-table [id=e5f90405883441318a549dfbb2dcb5ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:44.188063 29542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45cc41f0cb8146dca880e4c08d271794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:44.188557 29658 tablet_bootstrap.cc:492] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c: Bootstrap starting.
I20260812 16:32:44.188941 29658 tablet_bootstrap.cc:654] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:44.189242 29657 tablet_bootstrap.cc:492] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f: Bootstrap starting.
I20260812 16:32:44.189384 29379 tablet_service.cc:1467] Processing CreateTablet for tablet 45cc41f0cb8146dca880e4c08d271794 (DEFAULT_TABLE table=test-table [id=e5f90405883441318a549dfbb2dcb5ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:32:44.189500 29379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45cc41f0cb8146dca880e4c08d271794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:44.189555 29658 tablet_bootstrap.cc:492] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c: No bootstrap required, opened a new log
I20260812 16:32:44.189606 29658 ts_tablet_manager.cc:1397] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:44.189679 29657 tablet_bootstrap.cc:654] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:44.190238 29657 tablet_bootstrap.cc:492] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f: No bootstrap required, opened a new log
I20260812 16:32:44.190292 29657 ts_tablet_manager.cc:1397] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:44.190410 29657 raft_consensus.cc:348] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:44.190505 29657 raft_consensus.cc:374] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:44.190531 29657 raft_consensus.cc:729] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8585b614b6cc42c0b9cb217b143ae39f, State: Initialized, Role: FOLLOWER
I20260812 16:32:44.190578 29657 consensus_queue.cc:260] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:44.190692 29657 ts_tablet_manager.cc:1428] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:44.189756 29658 raft_consensus.cc:348] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:44.191048 29658 raft_consensus.cc:374] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:44.191073 29658 raft_consensus.cc:729] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d50f865d8894bb784ce0f6d7654011c, State: Initialized, Role: FOLLOWER
I20260812 16:32:44.191095 29661 tablet_bootstrap.cc:492] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5: Bootstrap starting.
I20260812 16:32:44.191118 29658 consensus_queue.cc:260] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:44.191129 29037 catalog_manager.cc:4841] TS 7d50f865d8894bb784ce0f6d7654011c (127.19.238.65:41743): tablet deedfcfd58d240edb019fac2777cdd90 (table test-table [id=baf7abf5cb864a5286d122ceeb08eed3]) successfully deleted
I20260812 16:32:44.191212 29658 ts_tablet_manager.cc:1428] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260812 16:32:44.191439 29661 tablet_bootstrap.cc:654] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:44.192067 29661 tablet_bootstrap.cc:492] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5: No bootstrap required, opened a new log
I20260812 16:32:44.192124 29661 ts_tablet_manager.cc:1397] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:44.192241 29661 raft_consensus.cc:348] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:44.192312 29661 raft_consensus.cc:374] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:44.192355 29661 raft_consensus.cc:729] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 976cd6755f3a4a17ba2e220f591ea9f5, State: Initialized, Role: FOLLOWER
I20260812 16:32:44.192414 29661 consensus_queue.cc:260] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:44.192539 29661 ts_tablet_manager.cc:1428] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:44.258455 29663 raft_consensus.cc:482] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:32:44.258591 29663 raft_consensus.cc:504] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:44.258927 29663 leader_election.cc:288] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 976cd6755f3a4a17ba2e220f591ea9f5 (127.19.238.66:46241), 8585b614b6cc42c0b9cb217b143ae39f (127.19.238.67:35269)
I20260812 16:32:44.263417 29741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:44.259185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:46241 (local address 127.19.238.65:47721)
0812 16:32:44.259504 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:44.259515 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:44.259580 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:44.259733 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:44.259734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:44.259736 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:44.259879 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:44.259882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.260875 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.260876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:44.262336 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:44.262339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.263169 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.263169 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:44.263258 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:44.263262 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:44.263306 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 16:32:44.263345 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":88,"thread_start_us":69,"threads_started":1}
I20260812 16:32:44.263437 29656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:44.259299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:47721 (local address 127.19.238.66:46241)
0812 16:32:44.259354 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:44.259357 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:44.259586 (+ 229us) server_negotiation.cc:407] Connection header received
0812 16:32:44.259650 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:44.259651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:44.259663 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:44.259720 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:44.259963 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.260747 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.262447 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.263029 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.263102 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:44.263111 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:44.263328 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 16:32:44.263369 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 16:32:44.263746 29402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45cc41f0cb8146dca880e4c08d271794" candidate_uuid: "7d50f865d8894bb784ce0f6d7654011c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" is_pre_election: true
I20260812 16:32:44.263875 29402 raft_consensus.cc:2454] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d50f865d8894bb784ce0f6d7654011c in term 0.
I20260812 16:32:44.263907 29740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:44.259185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:35269 (local address 127.19.238.65:44075)
0812 16:32:44.259326 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:44.259340 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:44.259443 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:44.259712 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:44.259713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:44.259715 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:32:44.259880 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:44.259882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.260875 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.260876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:44.262336 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:44.262339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.263584 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.263585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:44.263664 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:44.263668 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:32:44.263750 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 16:32:44.263790 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":77,"thread_start_us":70,"threads_started":1}
I20260812 16:32:44.263935 29655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:44.259309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:44075 (local address 127.19.238.67:35269)
0812 16:32:44.259363 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:44.259365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:44.259419 (+ 54us) server_negotiation.cc:407] Connection header received
0812 16:32:44.259580 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:44.259582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:44.259602 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:44.259679 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:32:44.259960 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.260747 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.262447 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.263021 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.263102 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:44.263111 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:44.263780 (+ 669us) server_negotiation.cc:299] Negotiation successful
0812 16:32:44.263867 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 16:32:44.264148 29154 leader_election.cc:302] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d50f865d8894bb784ce0f6d7654011c, 976cd6755f3a4a17ba2e220f591ea9f5; no voters:
I20260812 16:32:44.264227 29566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45cc41f0cb8146dca880e4c08d271794" candidate_uuid: "7d50f865d8894bb784ce0f6d7654011c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8585b614b6cc42c0b9cb217b143ae39f" is_pre_election: true
I20260812 16:32:44.264292 29663 raft_consensus.cc:2781] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:32:44.264349 29566 raft_consensus.cc:2454] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d50f865d8894bb784ce0f6d7654011c in term 0.
I20260812 16:32:44.264338 29663 raft_consensus.cc:482] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:32:44.264389 29663 raft_consensus.cc:3037] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:44.265141 29663 raft_consensus.cc:504] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:44.265293 29663 leader_election.cc:288] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c [CANDIDATE]: Term 1 election: Requested vote from peers 976cd6755f3a4a17ba2e220f591ea9f5 (127.19.238.66:46241), 8585b614b6cc42c0b9cb217b143ae39f (127.19.238.67:35269)
I20260812 16:32:44.265458 29566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45cc41f0cb8146dca880e4c08d271794" candidate_uuid: "7d50f865d8894bb784ce0f6d7654011c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8585b614b6cc42c0b9cb217b143ae39f"
I20260812 16:32:44.265471 29402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45cc41f0cb8146dca880e4c08d271794" candidate_uuid: "7d50f865d8894bb784ce0f6d7654011c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "976cd6755f3a4a17ba2e220f591ea9f5"
I20260812 16:32:44.265547 29402 raft_consensus.cc:3037] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:44.265547 29566 raft_consensus.cc:3037] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:44.266404 29402 raft_consensus.cc:2454] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d50f865d8894bb784ce0f6d7654011c in term 1.
I20260812 16:32:44.266479 29566 raft_consensus.cc:2454] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d50f865d8894bb784ce0f6d7654011c in term 1.
I20260812 16:32:44.266655 29156 leader_election.cc:302] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d50f865d8894bb784ce0f6d7654011c, 8585b614b6cc42c0b9cb217b143ae39f; no voters:
I20260812 16:32:44.266746 29663 raft_consensus.cc:2781] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:44.266855 29663 raft_consensus.cc:686] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c [term 1 LEADER]: Becoming Leader. State: Replica: 7d50f865d8894bb784ce0f6d7654011c, State: Running, Role: LEADER
I20260812 16:32:44.266940 29663 consensus_queue.cc:237] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } }
I20260812 16:32:44.267635 29053 catalog_manager.cc:5495] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d50f865d8894bb784ce0f6d7654011c (127.19.238.65). New cstate: current_term: 1 leader_uuid: "7d50f865d8894bb784ce0f6d7654011c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d50f865d8894bb784ce0f6d7654011c" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 } health_report { overall_health: UNKNOWN } } }
I20260812 16:32:44.288362 29744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:44.285751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35106 (local address 127.19.238.65:41743)
0812 16:32:44.285914 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:44.285916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:44.285921 (+ 5us) server_negotiation.cc:407] Connection header received
0812 16:32:44.285959 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:44.285959 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:44.285977 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:44.286069 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:44.286179 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.286744 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.287465 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.287696 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.287775 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:44.287956 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:44.288078 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:44.288162 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:44.288208 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 16:32:44.288230 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":105,"threads_started":1}
I20260812 16:32:44.293620 29749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:44.291004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:46241 (local address 127.19.238.65:34485)
0812 16:32:44.291230 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:44.291242 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:44.291304 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:44.291448 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:44.291449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:44.291450 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:32:44.291520 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:44.291522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.292174 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.292175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:44.292783 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:44.292785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.293154 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.293155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:44.293220 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:44.293243 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:32:44.293438 (+ 195us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:32:44.293441 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:32:44.293494 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 16:32:44.293537 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":111,"threads_started":1}
I20260812 16:32:44.293598 29656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:44.291111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:34485 (local address 127.19.238.66:46241)
0812 16:32:44.291201 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:44.291204 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:44.291293 (+ 89us) server_negotiation.cc:407] Connection header received
0812 16:32:44.291358 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:44.291359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:44.291371 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:44.291444 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:44.291599 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.292087 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.292878 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.293075 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.293152 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:44.293332 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:44.293370 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:44.293433 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:44.293498 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 16:32:44.293519 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 16:32:44.296084 29566 raft_consensus.cc:1261] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f [term 1 FOLLOWER]: Refusing update from remote peer 7d50f865d8894bb784ce0f6d7654011c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:44.296090 29402 raft_consensus.cc:1261] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 FOLLOWER]: Refusing update from remote peer 7d50f865d8894bb784ce0f6d7654011c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:32:44.296367 29742 consensus_queue.cc:1035] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c [LEADER]: Connected to new peer: Peer: permanent_uuid: "976cd6755f3a4a17ba2e220f591ea9f5" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 46241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:44.296522 29663 consensus_queue.cc:1035] T 45cc41f0cb8146dca880e4c08d271794 P 7d50f865d8894bb784ce0f6d7654011c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8585b614b6cc42c0b9cb217b143ae39f" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 35269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:32:44.304203 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:44.301668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:41743 (local address 127.19.238.66:37029)
0812 16:32:44.301711 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:44.301723 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:44.301764 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:44.301978 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:44.301999 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:44.302000 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:32:44.302052 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:44.302053 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.302714 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.302715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:44.303255 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:44.303257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.303676 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.303678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:44.303741 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:44.303748 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:32:44.303976 (+ 228us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:32:44.303979 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:32:44.304059 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 16:32:44.304111 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260812 16:32:44.304255 29744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:44.301770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:37029 (local address 127.19.238.65:41743)
0812 16:32:44.301837 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:44.301839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:44.301847 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:32:44.301870 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:44.301871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:44.301888 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:44.301978 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:32:44.302126 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.302640 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.303371 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:44.303597 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:44.303661 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:44.303853 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:32:44.303898 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:32:44.303974 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:44.304130 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 16:32:44.304173 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 16:32:44.311239 29052 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:44.311368 29052 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:32:44.312084 29052 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40ba34de68d34a16ad5ef4404de1d00a: Sending DeleteTablet for 3 replicas of tablet 45cc41f0cb8146dca880e4c08d271794
I20260812 16:32:44.312359 29542 tablet_service.cc:1514] Processing DeleteTablet for tablet 45cc41f0cb8146dca880e4c08d271794 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:44 UTC) from {username='oryx', principal='oryx/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:38062
I20260812 16:32:44.312378 29379 tablet_service.cc:1514] Processing DeleteTablet for tablet 45cc41f0cb8146dca880e4c08d271794 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:44 UTC) from {username='oryx', principal='oryx/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:40562
I20260812 16:32:44.312466 29683 tablet_replica.cc:330] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f: stopping tablet replica
I20260812 16:32:44.312484 29685 tablet_replica.cc:330] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5: stopping tablet replica
I20260812 16:32:44.312520 29683 raft_consensus.cc:2229] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:44.312530 29685 raft_consensus.cc:2229] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:32:44.312559 29683 raft_consensus.cc:2258] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:44.312565 29685 raft_consensus.cc:2258] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:32:44.312867 29213 tablet_service.cc:1514] Processing DeleteTablet for tablet 45cc41f0cb8146dca880e4c08d271794 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:32:44 UTC) from {username='oryx', principal='oryx/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:35104
I20260812 16:32:44.312932 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 29137
I20260812 16:32:44.313081 29685 ts_tablet_manager.cc:1905] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:44.315083 29685 ts_tablet_manager.cc:1918] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:32:44.315136 29685 log.cc:1196] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/45cc41f0cb8146dca880e4c08d271794
I20260812 16:32:44.315336 29685 ts_tablet_manager.cc:1939] T 45cc41f0cb8146dca880e4c08d271794 P 976cd6755f3a4a17ba2e220f591ea9f5: Deleting consensus metadata
I20260812 16:32:44.316709 29037 catalog_manager.cc:4841] TS 976cd6755f3a4a17ba2e220f591ea9f5 (127.19.238.66:46241): tablet 45cc41f0cb8146dca880e4c08d271794 (table test-table [id=e5f90405883441318a549dfbb2dcb5ea]) successfully deleted
I20260812 16:32:44.320147 29683 ts_tablet_manager.cc:1905] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:32:44.321497 29683 ts_tablet_manager.cc:1918] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:32:44.321583 29683 log.cc:1196] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/45cc41f0cb8146dca880e4c08d271794
W20260812 16:32:44.321720 29037 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.238.65:41743 (error 108)
I20260812 16:32:44.321839 29683 ts_tablet_manager.cc:1939] T 45cc41f0cb8146dca880e4c08d271794 P 8585b614b6cc42c0b9cb217b143ae39f: Deleting consensus metadata
I20260812 16:32:44.322209 29038 catalog_manager.cc:4841] TS 8585b614b6cc42c0b9cb217b143ae39f (127.19.238.67:35269): tablet 45cc41f0cb8146dca880e4c08d271794 (table test-table [id=e5f90405883441318a549dfbb2dcb5ea]) successfully deleted
I20260812 16:32:44.322423 29755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:44.322132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:41743 (local address 127.0.0.1:35114)
0812 16:32:44.322325 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:44.322374 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:41743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":48,"threads_started":1}
I20260812 16:32:44.322502 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 29305
W20260812 16:32:44.322552 29037 catalog_manager.cc:4584] TS 7d50f865d8894bb784ce0f6d7654011c (127.19.238.65:41743): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 45cc41f0cb8146dca880e4c08d271794: Network error: Client connection negotiation failed: client connection to 127.19.238.65:41743: connect: Connection refused (error 111)
I20260812 16:32:44.330459 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 29469
I20260812 16:32:44.338047 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 29017
2026-08-12T16:32:44Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2923 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:32:44 dist-test-slave-mvvj krb5kdc[29759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:32:44 dist-test-slave-mvvj krb5kdc[29759](info): set up 2 sockets
Aug 12 16:32:44 dist-test-slave-mvvj krb5kdc[29759](info): commencing operation
krb5kdc: starting...
W20260812 16:32:45.449118 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:32:45 dist-test-slave-mvvj krb5kdc[29759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:32:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:32:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:32:45.587438 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:40397
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:44919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:40397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:45.671396 29775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:45.671676 29775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:45.671720 29775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:45.673005 29775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:45.673094 29775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:45.673117 29775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:45.673139 29775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:45.673170 29775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:32:45.674774 29775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:44919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:40397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:40397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:45.675083 29775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:45.675338 29775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:45.680555 29785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:45.680555 29783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:45.680555 29782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:45.680884 29775 server_base.cc:1034] running on GCE node
I20260812 16:32:45.681185 29775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:45.681516 29775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:45.682708 29775 hybrid_clock.cc:648] HybridClock initialized: now 1786552365682700 us; error 52 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T16:32:45Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1344 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:32:45 dist-test-slave-mvvj krb5kdc[29793](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:32:45 dist-test-slave-mvvj krb5kdc[29793](info): set up 2 sockets
Aug 12 16:32:45 dist-test-slave-mvvj krb5kdc[29793](info): commencing operation
krb5kdc: starting...
W20260812 16:32:46.795639 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:32:46 dist-test-slave-mvvj krb5kdc[29793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:32:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:32:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:32:46.944207 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:37563
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:37563
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:47.029464 29808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:47.029724 29808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:47.029762 29808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:47.030951 29808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:47.031025 29808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:47.031049 29808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:47.031072 29808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:47.031103 29808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:32:47.032671 29808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:37563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:37563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:47.032980 29808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:47.033223 29808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:47.038280 29814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:47.038316 29815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:47.038317 29817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:47.038439 29808 server_base.cc:1034] running on GCE node
I20260812 16:32:47.038823 29808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:47.039084 29808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:47.040256 29808 hybrid_clock.cc:648] HybridClock initialized: now 1786552367040233 us; error 38 us; skew 500 ppm
Aug 12 16:32:47 dist-test-slave-mvvj krb5kdc[29793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:47.042837 29808 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:32:47.043347 29808 webserver.cc:460] Webserver started at http://127.19.238.126:40027/ using document root <none> and password file <none>
I20260812 16:32:47.043597 29808 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:47.043670 29808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:47.043823 29808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:47.044981 29808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "3c0bc4e02b6741cfa51e4a29ea12033b"
format_stamp: "Formatted at 2026-08-12 16:32:47 on dist-test-slave-mvvj"
server_key: "15de0533d6dc0c8fb90899fff59ac19e"
server_key_iv: "13ce03b0a6fd6bacbbf90dea0da11bfe"
server_key_version: "encryptionkey@0"
I20260812 16:32:47.045368 29808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "3c0bc4e02b6741cfa51e4a29ea12033b"
format_stamp: "Formatted at 2026-08-12 16:32:47 on dist-test-slave-mvvj"
server_key: "15de0533d6dc0c8fb90899fff59ac19e"
server_key_iv: "13ce03b0a6fd6bacbbf90dea0da11bfe"
server_key_version: "encryptionkey@0"
I20260812 16:32:47.046809 29808 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:32:47.047765 29827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:47.047984 29808 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:47.048106 29808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "3c0bc4e02b6741cfa51e4a29ea12033b"
format_stamp: "Formatted at 2026-08-12 16:32:47 on dist-test-slave-mvvj"
server_key: "15de0533d6dc0c8fb90899fff59ac19e"
server_key_iv: "13ce03b0a6fd6bacbbf90dea0da11bfe"
server_key_version: "encryptionkey@0"
I20260812 16:32:47.048194 29808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:47.076977 29808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:47.079510 29808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:47.079705 29808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:47.083874 29808 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:37563
I20260812 16:32:47.083930 29886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:37563 every 8 connection(s)
I20260812 16:32:47.084316 29808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:32:47.085322 29887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:32:47.087795 29887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b: Bootstrap starting.
I20260812 16:32:47.088397 29887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b: Neither blocks nor log segments found. Creating new log.
I20260812 16:32:47.088801 29887 log.cc:824] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:47.089603 29887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b: No bootstrap required, opened a new log
I20260812 16:32:47.090916 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 29808
I20260812 16:32:47.091058 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:32:47.091265 20409 external_mini_cluster.cc:1428] Setting key 3ff42f19fcf626a59322b3d5dfb0ebb4
I20260812 16:32:47.091394 29887 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0bc4e02b6741cfa51e4a29ea12033b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37563 } }
I20260812 16:32:47.091605 29887 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:32:47.091655 29887 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c0bc4e02b6741cfa51e4a29ea12033b, State: Initialized, Role: FOLLOWER
I20260812 16:32:47.091915 29887 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0bc4e02b6741cfa51e4a29ea12033b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37563 } }
I20260812 16:32:47.092028 29887 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:32:47.092087 29887 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:32:47.092151 29887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:32:47.093364 29887 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0bc4e02b6741cfa51e4a29ea12033b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37563 } }
I20260812 16:32:47.093505 29887 leader_election.cc:302] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c0bc4e02b6741cfa51e4a29ea12033b; no voters:
I20260812 16:32:47.093693 29887 leader_election.cc:288] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:32:47.093744 29892 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:32:47.093937 29892 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [term 1 LEADER]: Becoming Leader. State: Replica: 3c0bc4e02b6741cfa51e4a29ea12033b, State: Running, Role: LEADER
I20260812 16:32:47.093973 29887 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:47.094062 29892 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0bc4e02b6741cfa51e4a29ea12033b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37563 } }
I20260812 16:32:47.094408 29892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c0bc4e02b6741cfa51e4a29ea12033b. Latest consensus state: current_term: 1 leader_uuid: "3c0bc4e02b6741cfa51e4a29ea12033b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0bc4e02b6741cfa51e4a29ea12033b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37563 } } }
I20260812 16:32:47.094636 29892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:47.094564 29894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c0bc4e02b6741cfa51e4a29ea12033b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0bc4e02b6741cfa51e4a29ea12033b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37563 } } }
I20260812 16:32:47.094868 29894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:47.095070 29903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:32:47.095600 29903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:32:47.097596 29903 catalog_manager.cc:1347] Generated new cluster ID: 40aff16bceac438889b857d83387d476
I20260812 16:32:47.097679 29903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:32:47 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:47.104048 29891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:47.091920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59804 (local address 127.19.238.126:37563)
0812 16:32:47.092143 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:47.092160 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:47.092184 (+ 24us) server_negotiation.cc:407] Connection header received
0812 16:32:47.092329 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:47.092335 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:47.092381 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:47.092527 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:47.093366 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:47.094875 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:47.096145 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:47.096650 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:47.096779 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:47.099710 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:47.099715 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:47.099715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:47.099729 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:47.101765 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:47.102197 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:47.102200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:47.102200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:47.102328 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:47.102554 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:47.102555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:47.102555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:47.102869 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:47.103584 (+ 715us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:47.103666 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 16:32:47.103810 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":137,"threads_started":1}
I20260812 16:32:47.116894 29903 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:32:47.117519 29903 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:47.126209 29903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b: Generated new TSK 0
I20260812 16:32:47.126442 29903 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
I20260812 16:32:47.163527 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:37563
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:47.248773 29916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:47.248983 29916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:47.249008 29916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:47.250128 29916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:47.250162 29916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:47.250188 29916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:32:47.251683 29916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:37563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:47.251941 29916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:47.252157 29916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:47.257316 29924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:47.257333 29926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:47.257333 29922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:47.257632 29916 server_base.cc:1034] running on GCE node
I20260812 16:32:47.257920 29916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:47.258149 29916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:47.259306 29916 hybrid_clock.cc:648] HybridClock initialized: now 1786552367259285 us; error 35 us; skew 500 ppm
Aug 12 16:32:47 dist-test-slave-mvvj krb5kdc[29793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:47.262198 29916 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:32:47.262720 29916 webserver.cc:460] Webserver started at http://127.19.238.65:37639/ using document root <none> and password file <none>
I20260812 16:32:47.262966 29916 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:47.263038 29916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:47.263204 29916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:47.264415 29916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "9ead04e678d4418d889906fdcd2c0c3c"
format_stamp: "Formatted at 2026-08-12 16:32:47 on dist-test-slave-mvvj"
server_key: "098e26b5b8e8116cb7ec5de87944ae18"
server_key_iv: "d0b395e8b7e70df2204fd2d0faa6020c"
server_key_version: "encryptionkey@0"
I20260812 16:32:47.264815 29916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "9ead04e678d4418d889906fdcd2c0c3c"
format_stamp: "Formatted at 2026-08-12 16:32:47 on dist-test-slave-mvvj"
server_key: "098e26b5b8e8116cb7ec5de87944ae18"
server_key_iv: "d0b395e8b7e70df2204fd2d0faa6020c"
server_key_version: "encryptionkey@0"
I20260812 16:32:47.266187 29916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:32:47.267089 29933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:47.267319 29916 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:47.267488 29916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "9ead04e678d4418d889906fdcd2c0c3c"
format_stamp: "Formatted at 2026-08-12 16:32:47 on dist-test-slave-mvvj"
server_key: "098e26b5b8e8116cb7ec5de87944ae18"
server_key_iv: "d0b395e8b7e70df2204fd2d0faa6020c"
server_key_version: "encryptionkey@0"
I20260812 16:32:47.267575 29916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:47.290664 29916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:47.293272 29916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:47.293479 29916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:47.293872 29916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:47.294262 29916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:47.294327 29916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:47.294409 29916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:47.294476 29916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:47.301399 29916 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:42281
I20260812 16:32:47.301441 30061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:42281 every 8 connection(s)
I20260812 16:32:47.301826 29916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
Aug 12 16:32:47 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:47.309747 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 29916
I20260812 16:32:47.309888 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:32:47.310153 20409 external_mini_cluster.cc:1428] Setting key 23a40c9f92c23b469dc677c2536e8432
I20260812 16:32:47.314079 29891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:47.302699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:36049 (local address 127.19.238.126:37563)
0812 16:32:47.302793 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:47.302795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:47.302991 (+ 196us) server_negotiation.cc:407] Connection header received
0812 16:32:47.303594 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:47.303595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:47.303610 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:47.303661 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:47.304874 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:47.305483 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:47.306727 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:47.306963 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:47.307030 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:47.309726 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:47.309728 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:47.309728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:47.309739 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:47.312077 (+ 2338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:47.312636 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:47.312638 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:47.312638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:47.312717 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:47.313129 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:47.313131 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:47.313131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:47.313313 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:47.313366 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:47.313813 (+ 447us) server_negotiation.cc:299] Negotiation successful
0812 16:32:47.313938 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 16:32:47.314121 30065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:47.302640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.65:36049)
0812 16:32:47.302841 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:47.302890 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:47.303493 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:47.303732 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:47.303736 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:47.304510 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:47.304736 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:47.304742 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:47.305607 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:47.305608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:47.306623 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:47.306626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:47.307055 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:47.307056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:47.307171 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:47.307811 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:47.307820 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:47.309585 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:47.312218 (+ 2633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:47.312227 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:47.312235 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:47.312535 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:47.312815 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:47.312816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:47.312817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:47.313017 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:47.313423 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:47.313431 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:47.313633 (+ 202us) client_negotiation.cc:769] Sending connection context
0812 16:32:47.313758 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 16:32:47.313940 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":111,"threads_started":1}
I20260812 16:32:47.314769 30062 heartbeater.cc:347] Connected to a master server at 127.19.238.126:37563
I20260812 16:32:47.314889 30062 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:47.315089 30062 heartbeater.cc:511] Master 127.19.238.126:37563 requested a full tablet report, sending...
I20260812 16:32:47.315781 29847 ts_manager.cc:194] Registered new tserver with Master: 9ead04e678d4418d889906fdcd2c0c3c (127.19.238.65:42281)
I20260812 16:32:47.316949 29847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:36049
WARNING: no policy specified for kudu/127.19.238.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
I20260812 16:32:47.343993 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:37563
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:47.428931 30069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:47.429147 30069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:47.429171 30069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:47.430418 30069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:47.430469 30069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:47.430496 30069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:32:47.432041 30069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:37563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:47.432233 30069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:47.432554 30069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:47.437850 30075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:47.437882 30074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:47.437857 30077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:47.438186 30069 server_base.cc:1034] running on GCE node
I20260812 16:32:47.438380 30069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:47.438609 30069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:47.439757 30069 hybrid_clock.cc:648] HybridClock initialized: now 1786552367439742 us; error 31 us; skew 500 ppm
Aug 12 16:32:47 dist-test-slave-mvvj krb5kdc[29793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:47.442507 30069 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:32:47.443017 30069 webserver.cc:460] Webserver started at http://127.19.238.66:44987/ using document root <none> and password file <none>
I20260812 16:32:47.443234 30069 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:47.443276 30069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:47.443369 30069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:47.444568 30069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "42f96bb1245c4d1f8da37ee56d7f1778"
format_stamp: "Formatted at 2026-08-12 16:32:47 on dist-test-slave-mvvj"
server_key: "9a4786196215d2e5f523891eb851d252"
server_key_iv: "8aa9f397d3df0f9f1fa83bec37d66afa"
server_key_version: "encryptionkey@0"
I20260812 16:32:47.444885 30069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "42f96bb1245c4d1f8da37ee56d7f1778"
format_stamp: "Formatted at 2026-08-12 16:32:47 on dist-test-slave-mvvj"
server_key: "9a4786196215d2e5f523891eb851d252"
server_key_iv: "8aa9f397d3df0f9f1fa83bec37d66afa"
server_key_version: "encryptionkey@0"
I20260812 16:32:47.446190 30069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:32:47.446926 30085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:47.447129 30069 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:32:47.447216 30069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "42f96bb1245c4d1f8da37ee56d7f1778"
format_stamp: "Formatted at 2026-08-12 16:32:47 on dist-test-slave-mvvj"
server_key: "9a4786196215d2e5f523891eb851d252"
server_key_iv: "8aa9f397d3df0f9f1fa83bec37d66afa"
server_key_version: "encryptionkey@0"
I20260812 16:32:47.447270 30069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:47.459259 30069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:47.462363 30069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:47.462617 30069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:47.462930 30069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:47.463406 30069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:47.463466 30069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:47.463501 30069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:47.463522 30069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:47.470616 30069 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:45139
I20260812 16:32:47.470708 30216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:45139 every 8 connection(s)
I20260812 16:32:47.471066 30069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
Aug 12 16:32:47 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:47.480331 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 30069
I20260812 16:32:47.480517 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:32:47.480765 20409 external_mini_cluster.cc:1428] Setting key b06dac33483ff8cfdf09a334927bf878
I20260812 16:32:47.481302 29891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:47.471933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:55183 (local address 127.19.238.126:37563)
0812 16:32:47.472012 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:47.472015 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:47.472234 (+ 219us) server_negotiation.cc:407] Connection header received
0812 16:32:47.472812 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:47.472813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:47.472824 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:47.472882 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:47.473863 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:47.474370 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:47.475386 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:47.475591 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:47.475646 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:47.478210 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:47.478211 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:47.478211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:47.478218 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:47.479510 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:47.480040 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:47.480055 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:47.480056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:47.480114 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:47.480451 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:47.480452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:47.480452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:47.480625 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:47.480685 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:47.481097 (+ 412us) server_negotiation.cc:299] Negotiation successful
0812 16:32:47.481217 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 16:32:47.481541 30220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:47.471905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.66:55183)
0812 16:32:47.472103 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:47.472144 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:47.472689 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:47.472949 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:47.472952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:47.473569 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:47.473773 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:47.473779 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:47.474458 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:47.474459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:47.475302 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:47.475305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:47.475677 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:47.475678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:47.475792 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:47.476467 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:47.476476 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:47.478103 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:47.479622 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:47.479629 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:47.479631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:47.479967 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:47.480189 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:47.480190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:47.480191 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:47.480351 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:47.480685 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:47.480694 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:47.480880 (+ 186us) client_negotiation.cc:769] Sending connection context
0812 16:32:47.481001 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 16:32:47.481203 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":112,"threads_started":1}
I20260812 16:32:47.482146 30218 heartbeater.cc:347] Connected to a master server at 127.19.238.126:37563
I20260812 16:32:47.482259 30218 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:47.482481 30218 heartbeater.cc:511] Master 127.19.238.126:37563 requested a full tablet report, sending...
I20260812 16:32:47.482944 29847 ts_manager.cc:194] Registered new tserver with Master: 42f96bb1245c4d1f8da37ee56d7f1778 (127.19.238.66:45139)
I20260812 16:32:47.483476 29847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:55183
WARNING: no policy specified for kudu/127.19.238.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
I20260812 16:32:47.510888 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:37563
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:47.594285 30223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:47.594527 30223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:47.594578 30223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:47.596014 30223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:47.596058 30223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:47.596086 30223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:32:47.597591 30223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:37563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:47.597846 30223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:47.598085 30223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:47.603099 30231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:47.603163 30229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:47.603099 30228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:47.603438 30223 server_base.cc:1034] running on GCE node
I20260812 16:32:47.603969 30223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:47.604187 30223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:47.605433 30223 hybrid_clock.cc:648] HybridClock initialized: now 1786552367605411 us; error 35 us; skew 500 ppm
Aug 12 16:32:47 dist-test-slave-mvvj krb5kdc[29793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:47.608280 30223 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:32:47.608790 30223 webserver.cc:460] Webserver started at http://127.19.238.67:39773/ using document root <none> and password file <none>
I20260812 16:32:47.608994 30223 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:47.609035 30223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:47.609128 30223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:32:47.610276 30223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "26de6b86e5c845e59b8bb54df648ff63"
format_stamp: "Formatted at 2026-08-12 16:32:47 on dist-test-slave-mvvj"
server_key: "52c5cf51d9689237c578384e64d6ac45"
server_key_iv: "e4a61f865ea7c927a62e0bc9941811e7"
server_key_version: "encryptionkey@0"
I20260812 16:32:47.610610 30223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "26de6b86e5c845e59b8bb54df648ff63"
format_stamp: "Formatted at 2026-08-12 16:32:47 on dist-test-slave-mvvj"
server_key: "52c5cf51d9689237c578384e64d6ac45"
server_key_iv: "e4a61f865ea7c927a62e0bc9941811e7"
server_key_version: "encryptionkey@0"
I20260812 16:32:47.612063 30223 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:32:47.612923 30239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:47.613113 30223 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:47.613226 30223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "26de6b86e5c845e59b8bb54df648ff63"
format_stamp: "Formatted at 2026-08-12 16:32:47 on dist-test-slave-mvvj"
server_key: "52c5cf51d9689237c578384e64d6ac45"
server_key_iv: "e4a61f865ea7c927a62e0bc9941811e7"
server_key_version: "encryptionkey@0"
I20260812 16:32:47.613368 30223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:47.643167 30223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:47.645650 30223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:47.645843 30223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:47.646121 30223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:47.646487 30223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:47.646548 30223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:47.646591 30223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:47.646641 30223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:47.653313 30223 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:35195
I20260812 16:32:47.653348 30371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:35195 every 8 connection(s)
I20260812 16:32:47.653707 30223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:32:47.657105 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 30223
I20260812 16:32:47.657195 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:32:47.657387 20409 external_mini_cluster.cc:1428] Setting key 78efe57bf342b81def5212644efc866f
Aug 12 16:32:47 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:32:47.663129 29891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:47.654460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:46179 (local address 127.19.238.126:37563)
0812 16:32:47.654536 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:47.654539 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:47.654776 (+ 237us) server_negotiation.cc:407] Connection header received
0812 16:32:47.655253 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:47.655253 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:47.655265 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:47.655329 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:47.656368 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:47.656859 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:47.658145 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:47.658422 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:47.658533 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:47.660633 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:47.660635 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:47.660635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:47.660642 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:47.661884 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:47.662256 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:47.662257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:47.662257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:47.662308 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:47.662601 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:47.662602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:47.662602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:47.662707 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:47.662759 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:47.663003 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 16:32:47.663065 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 16:32:47.663417 30375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:47.654485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.67:46179)
0812 16:32:47.654647 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:47.654684 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:47.655168 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:47.655386 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:47.655390 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:47.656063 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:47.656278 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:47.656283 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:47.656957 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:47.656958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:47.658087 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:47.658091 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:47.658543 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:47.658544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:47.658646 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:47.659130 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:47.659138 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:47.660548 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:32:47.661961 (+ 1413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:47.661967 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:47.661968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:47.662189 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:47.662419 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:32:47.662420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:32:47.662421 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:32:47.662533 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:32:47.662765 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:32:47.662769 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:32:47.662885 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 16:32:47.662970 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 16:32:47.663107 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
I20260812 16:32:47.663914 30372 heartbeater.cc:347] Connected to a master server at 127.19.238.126:37563
I20260812 16:32:47.664027 30372 heartbeater.cc:464] Registering TS with master...
I20260812 16:32:47.664206 30372 heartbeater.cc:511] Master 127.19.238.126:37563 requested a full tablet report, sending...
I20260812 16:32:47.664597 29847 ts_manager.cc:194] Registered new tserver with Master: 26de6b86e5c845e59b8bb54df648ff63 (127.19.238.67:35195)
I20260812 16:32:47.664939 29847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:46179
I20260812 16:32:47.669605 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.65.keytab.
Entry for principal oryx/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.65.keytab.
WARNING: no policy specified for oryx/127.19.238.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.66.keytab.
Entry for principal oryx/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.66.keytab.
WARNING: no policy specified for oryx/127.19.238.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.238.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.67.keytab.
Entry for principal oryx/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.67.keytab.
I20260812 16:32:47.751994 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 29916
I20260812 16:32:47.771986 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 30069
I20260812 16:32:47.782810 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 30223
I20260812 16:32:47.797221 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 29808
I20260812 16:32:47.804577 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:37563
--webserver_interface=127.19.238.126
--webserver_port=40027
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:37563
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:47.895135 30382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:47.895357 30382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:47.895381 30382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:47.896612 30382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:32:47.896662 30382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:47.896677 30382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:47.896689 30382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:32:47.896701 30382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:32:47.898180 30382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:37563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:37563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=40027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:47.898406 30382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:47.898605 30382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:47.903689 30388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:47.903682 30391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:47.903682 30387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:47.904003 30382 server_base.cc:1034] running on GCE node
I20260812 16:32:47.904249 30382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:47.904592 30382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:47.905742 30382 hybrid_clock.cc:648] HybridClock initialized: now 1786552367905714 us; error 39 us; skew 500 ppm
Aug 12 16:32:47 dist-test-slave-mvvj krb5kdc[29793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552367, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:47.908965 30382 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:32:47.909459 30382 webserver.cc:460] Webserver started at http://127.19.238.126:40027/ using document root <none> and password file <none>
I20260812 16:32:47.909686 30382 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:47.909746 30382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:47.911273 30382 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:47.912086 30401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:47.912369 30382 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:47.912494 30382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "3c0bc4e02b6741cfa51e4a29ea12033b"
format_stamp: "Formatted at 2026-08-12 16:32:47 on dist-test-slave-mvvj"
server_key: "15de0533d6dc0c8fb90899fff59ac19e"
server_key_iv: "13ce03b0a6fd6bacbbf90dea0da11bfe"
server_key_version: "encryptionkey@0"
I20260812 16:32:47.912895 30382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:47.971626 30382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:47.974103 30382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:47.974263 30382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:47.978610 30382 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:37563
I20260812 16:32:47.978647 30461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:37563 every 8 connection(s)
I20260812 16:32:47.979064 30382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:32:47.980679 30462 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:32:47.981380 30462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b: Bootstrap starting.
I20260812 16:32:47.981899 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 30382
I20260812 16:32:47.982244 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:42281
--local_ip_for_outbound_sockets=127.19.238.65
--tserver_master_addrs=127.19.238.126:37563
--webserver_port=37639
--webserver_interface=127.19.238.65
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.238.65
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:32:47.990896 30462 log.cc:824] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b: Log is configured to *not* fsync() on all Append() calls
I20260812 16:32:47.992969 30462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:32:47.993309 30462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b: Bootstrap complete.
I20260812 16:32:47.995796 30462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0bc4e02b6741cfa51e4a29ea12033b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37563 } }
I20260812 16:32:47.996146 30462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c0bc4e02b6741cfa51e4a29ea12033b, State: Initialized, Role: FOLLOWER
I20260812 16:32:47.996340 30462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0bc4e02b6741cfa51e4a29ea12033b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37563 } }
I20260812 16:32:47.996409 30462 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:32:47.996474 30462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:32:47.996521 30462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [term 1 FOLLOWER]: Advancing to term 2
I20260812 16:32:47.997593 30462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0bc4e02b6741cfa51e4a29ea12033b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37563 } }
I20260812 16:32:47.997722 30462 leader_election.cc:302] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c0bc4e02b6741cfa51e4a29ea12033b; no voters:
I20260812 16:32:47.997931 30462 leader_election.cc:288] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 16:32:47.997972 30466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [term 2 FOLLOWER]: Leader election won for term 2
I20260812 16:32:47.998091 30466 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [term 2 LEADER]: Becoming Leader. State: Replica: 3c0bc4e02b6741cfa51e4a29ea12033b, State: Running, Role: LEADER
I20260812 16:32:47.998190 30466 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0bc4e02b6741cfa51e4a29ea12033b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37563 } }
I20260812 16:32:47.998235 30462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:32:47.998505 30467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3c0bc4e02b6741cfa51e4a29ea12033b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0bc4e02b6741cfa51e4a29ea12033b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37563 } } }
I20260812 16:32:47.998601 30467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:47.998500 30466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c0bc4e02b6741cfa51e4a29ea12033b. Latest consensus state: current_term: 2 leader_uuid: "3c0bc4e02b6741cfa51e4a29ea12033b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0bc4e02b6741cfa51e4a29ea12033b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37563 } } }
I20260812 16:32:47.998819 30466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b [sys.catalog]: This master's current role is: LEADER
I20260812 16:32:47.999099 30471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:32:47.999651 30471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:32:47.999850 30471 catalog_manager.cc:1259] Loaded cluster ID: 40aff16bceac438889b857d83387d476
I20260812 16:32:48.000082 30471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:32:48.000936 30471 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:32:48.001147 30471 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3c0bc4e02b6741cfa51e4a29ea12033b: Loaded TSK: 0
I20260812 16:32:48.001368 30471 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 16:32:48.081526 30464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:48.081753 30464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:48.081779 30464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:48.083045 30464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:48.083094 30464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:48.083123 30464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:32:48.084697 30464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.65.keytab
--principal=oryx/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:42281
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=37639
--tserver_master_addrs=127.19.238.126:37563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:48.084949 30464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:48.085224 30464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:48.090823 30493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:48.091034 30491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:48.090993 30464 server_base.cc:1034] running on GCE node
W20260812 16:32:48.090818 30490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:48.091373 30464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:48.091653 30464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:48.092862 30464 hybrid_clock.cc:648] HybridClock initialized: now 1786552368092842 us; error 38 us; skew 500 ppm
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552368, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:48.095614 30464 init.cc:377] Logged in from keytab as oryx/127.19.238.65@KRBTEST.COM (short username oryx)
I20260812 16:32:48.096172 30464 webserver.cc:460] Webserver started at http://127.19.238.65:37639/ using document root <none> and password file <none>
I20260812 16:32:48.096421 30464 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:48.096506 30464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:48.098065 30464 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:32:48.098892 30502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:48.099072 30464 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:32:48.099191 30464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "9ead04e678d4418d889906fdcd2c0c3c"
format_stamp: "Formatted at 2026-08-12 16:32:47 on dist-test-slave-mvvj"
server_key: "098e26b5b8e8116cb7ec5de87944ae18"
server_key_iv: "d0b395e8b7e70df2204fd2d0faa6020c"
server_key_version: "encryptionkey@0"
I20260812 16:32:48.099702 30464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:48.120096 30464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:48.123004 30464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:48.123210 30464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:48.123541 30464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:48.123987 30464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:48.124046 30464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:48.124083 30464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:48.124110 30464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:48.130788 30464 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:42281
I20260812 16:32:48.130846 30630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:42281 every 8 connection(s)
I20260812 16:32:48.131238 30464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:32:48.139212 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 30464
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:48.139573 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:45139
--local_ip_for_outbound_sockets=127.19.238.66
--tserver_master_addrs=127.19.238.126:37563
--webserver_port=44987
--webserver_interface=127.19.238.66
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.238.66
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:48.141276 30633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:48.132192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.65:37433)
0812 16:32:48.132392 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:48.132446 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:48.132983 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:48.133486 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:48.133489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:48.134106 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:48.134316 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:48.134321 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.136138 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.136138 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:48.136975 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:48.136978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.137785 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.137785 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:48.137893 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:48.138525 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:48.138534 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:48.141046 (+ 2512us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
W20260812 16:32:48.141337 30634 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:48.132267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:37433 (local address 127.19.238.126:37563)
0812 16:32:48.132508 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:48.132517 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:48.132569 (+ 52us) server_negotiation.cc:407] Connection header received
0812 16:32:48.133214 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:48.133221 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:48.133280 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:48.133425 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:48.134522 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.136021 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.137079 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.137688 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.137803 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:48.141184 (+ 3381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.65:37433: BlockingRecv error: recv got EOF from 127.19.238.65:37433 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":100,"threads_started":1}
W20260812 16:32:48.141894 30631 heartbeater.cc:644] Failed to heartbeat to 127.19.238.126:37563 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.238.126:37563: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:48.146703 30633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:48.142156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.65:56109)
0812 16:32:48.142261 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:48.142271 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:48.142318 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:48.142484 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:48.142485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:48.142625 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:48.142674 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:48.142675 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.143348 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.143349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:48.143944 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:48.143946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.144327 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.144328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:48.144398 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:48.144858 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:48.144863 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:48.146617 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89}
W20260812 16:32:48.146781 30634 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:48.142215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:56109 (local address 127.19.238.126:37563)
0812 16:32:48.142253 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:48.142255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:48.142302 (+ 47us) server_negotiation.cc:407] Connection header received
0812 16:32:48.142378 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:48.142379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:48.142393 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:48.142466 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:48.142759 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.143259 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.144029 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.144223 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.144329 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:48.146705 (+ 2376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.65:56109: BlockingRecv error: recv got EOF from 127.19.238.65:56109 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:48.151556 30633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:48.147341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.65:36925)
0812 16:32:48.147385 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:48.147396 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:48.147434 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:48.147559 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:48.147561 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:48.147690 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:48.147717 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:48.147718 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.148377 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.148378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:48.148863 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:48.148865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.149253 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.149254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:48.149306 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:48.149757 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:48.149763 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:48.151466 (+ 1703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 16:32:48.151611 30634 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:48.147387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:36925 (local address 127.19.238.126:37563)
0812 16:32:48.147434 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:48.147436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:48.147444 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:32:48.147492 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:48.147492 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:48.147502 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:48.147547 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:48.147793 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.148290 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.148981 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.149178 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.149255 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:48.151544 (+ 2289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.65:36925: BlockingRecv error: recv got EOF from 127.19.238.65:36925 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 16:32:48.151958 30631 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 16:32:48.229450 30635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:48.229753 30635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:48.229779 30635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:48.230937 30635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:48.230973 30635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:48.230999 30635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:32:48.232414 30635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.66.keytab
--principal=oryx/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:45139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=44987
--tserver_master_addrs=127.19.238.126:37563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:48.232615 30635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:48.232810 30635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:48.238065 30643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:48.238119 30635 server_base.cc:1034] running on GCE node
W20260812 16:32:48.238062 30641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:48.238059 30640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:48.238613 30635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:48.238895 30635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:48.240038 30635 hybrid_clock.cc:648] HybridClock initialized: now 1786552368240005 us; error 48 us; skew 500 ppm
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552368, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:48.242707 30635 init.cc:377] Logged in from keytab as oryx/127.19.238.66@KRBTEST.COM (short username oryx)
I20260812 16:32:48.243227 30635 webserver.cc:460] Webserver started at http://127.19.238.66:44987/ using document root <none> and password file <none>
I20260812 16:32:48.243467 30635 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:48.243556 30635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:48.245153 30635 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:48.245916 30650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:48.246145 30635 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:48.246263 30635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "42f96bb1245c4d1f8da37ee56d7f1778"
format_stamp: "Formatted at 2026-08-12 16:32:47 on dist-test-slave-mvvj"
server_key: "9a4786196215d2e5f523891eb851d252"
server_key_iv: "8aa9f397d3df0f9f1fa83bec37d66afa"
server_key_version: "encryptionkey@0"
I20260812 16:32:48.246682 30635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:48.259948 30635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:48.262230 30635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:48.262414 30635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:48.262681 30635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:48.263033 30635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:48.263091 30635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:48.263141 30635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:48.263165 30635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:48.269510 30635 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:45139
I20260812 16:32:48.269577 30777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:45139 every 8 connection(s)
I20260812 16:32:48.269982 30635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:48.276705 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 30635
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:48.277016 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:35195
--local_ip_for_outbound_sockets=127.19.238.67
--tserver_master_addrs=127.19.238.126:37563
--webserver_port=39773
--webserver_interface=127.19.238.67
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.238.67
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:32:48.277716 30634 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:48.270849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:60537 (local address 127.19.238.126:37563)
0812 16:32:48.270911 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:48.270913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:48.271115 (+ 202us) server_negotiation.cc:407] Connection header received
0812 16:32:48.271596 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:48.271597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:48.271611 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:48.271674 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:48.272719 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.273220 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.274236 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.274429 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.274489 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:48.277640 (+ 3151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.66:60537: BlockingRecv error: recv got EOF from 127.19.238.66:60537 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 16:32:48.277765 30780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:48.270803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.66:60537)
0812 16:32:48.270975 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:48.271020 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:48.271510 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:48.271729 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:48.271732 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:48.272403 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:48.272602 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:48.272607 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.273323 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.273325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:48.274147 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:48.274150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.274510 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.274511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:48.274621 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:48.275165 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:48.275174 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:48.277554 (+ 2380us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
W20260812 16:32:48.278046 30778 heartbeater.cc:644] Failed to heartbeat to 127.19.238.126:37563 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.238.126:37563: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:48.283000 30780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:48.278344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.66:45537)
0812 16:32:48.278525 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:48.278535 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:48.278605 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:48.278759 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:48.278760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:48.278910 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:48.278957 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:48.278959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.279595 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.279596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:48.280169 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:48.280171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.280533 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.280534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:48.280613 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:48.281061 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:48.281071 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:48.282884 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88}
W20260812 16:32:48.283036 30634 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:48.278405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:45537 (local address 127.19.238.126:37563)
0812 16:32:48.278448 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:48.278450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:48.278591 (+ 141us) server_negotiation.cc:407] Connection header received
0812 16:32:48.278672 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:48.278673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:48.278684 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:48.278757 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:48.279043 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.279513 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.280264 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.280451 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.280523 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:48.282991 (+ 2468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.66:45537: BlockingRecv error: recv got EOF from 127.19.238.66:45537 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:48.287809 30780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:48.283378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.66:42857)
0812 16:32:48.283436 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:48.283476 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:48.283534 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:48.283679 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:48.283679 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:48.283805 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:48.283872 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:48.283873 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.284475 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.284476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:48.284958 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:48.284959 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.285342 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.285343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:48.285398 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:48.285834 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:48.285839 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:48.287713 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 16:32:48.287925 30634 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:48.283439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:42857 (local address 127.19.238.126:37563)
0812 16:32:48.283476 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:48.283478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:48.283535 (+ 57us) server_negotiation.cc:407] Connection header received
0812 16:32:48.283595 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:48.283596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:48.283605 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:48.283675 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:48.283941 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.284392 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.285056 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.285246 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.285330 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:48.287815 (+ 2485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.66:42857: BlockingRecv error: recv got EOF from 127.19.238.66:42857 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 16:32:48.288163 30778 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 16:32:48.362178 30781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:32:48.362478 30781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:32:48.362519 30781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:32:48.363739 30781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:32:48.363790 30781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:32:48.363816 30781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:32:48.365393 30781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/krb5kdc/oryx_127.19.238.67.keytab
--principal=oryx/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:35195
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=39773
--tserver_master_addrs=127.19.238.126:37563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:32:48.365687 30781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:32:48.365958 30781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:32:48.371359 30789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:48.371323 30786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:32:48.371312 30787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:32:48.371600 30781 server_base.cc:1034] running on GCE node
I20260812 16:32:48.371971 30781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:32:48.372257 30781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:32:48.373428 30781 hybrid_clock.cc:648] HybridClock initialized: now 1786552368373414 us; error 31 us; skew 500 ppm
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552368, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:32:48.376585 30781 init.cc:377] Logged in from keytab as oryx/127.19.238.67@KRBTEST.COM (short username oryx)
I20260812 16:32:48.377184 30781 webserver.cc:460] Webserver started at http://127.19.238.67:39773/ using document root <none> and password file <none>
I20260812 16:32:48.377408 30781 fs_manager.cc:359] Metadata directory not provided
I20260812 16:32:48.377465 30781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:32:48.378950 30781 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:32:48.379573 30797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:48.379809 30781 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:32:48.379942 30781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "26de6b86e5c845e59b8bb54df648ff63"
format_stamp: "Formatted at 2026-08-12 16:32:47 on dist-test-slave-mvvj"
server_key: "52c5cf51d9689237c578384e64d6ac45"
server_key_iv: "e4a61f865ea7c927a62e0bc9941811e7"
server_key_version: "encryptionkey@0"
I20260812 16:32:48.380352 30781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:32:48.391541 30781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:32:48.393996 30781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:32:48.394134 30781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:32:48.394331 30781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:32:48.394600 30781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:32:48.394630 30781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:48.394654 30781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:32:48.394667 30781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:32:48.401098 30781 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:35195
I20260812 16:32:48.401165 30927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:35195 every 8 connection(s)
I20260812 16:32:48.401517 30781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:32:48.403185 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 30781
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 16:32:48.410008 30634 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:48.402407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:51453 (local address 127.19.238.126:37563)
0812 16:32:48.402501 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:48.402504 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:48.402629 (+ 125us) server_negotiation.cc:407] Connection header received
0812 16:32:48.403226 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:48.403228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:48.403242 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:48.403319 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:48.404604 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.405129 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.406240 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.406444 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.406513 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:48.409948 (+ 3435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.67:51453: BlockingRecv error: recv got EOF from 127.19.238.67:51453 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 16:32:48.410085 30930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:48.402312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.67:51453)
0812 16:32:48.402471 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:48.402519 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:48.403116 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:48.403470 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:48.403474 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:48.404293 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:48.404492 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:48.404497 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.405225 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.405226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:48.406156 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:48.406158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.406538 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.406539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:48.406698 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:48.407286 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:48.407294 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:48.409874 (+ 2580us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
W20260812 16:32:48.410369 30928 heartbeater.cc:644] Failed to heartbeat to 127.19.238.126:37563 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.238.126:37563: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
I20260812 16:32:48.411646 30932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:48.403765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59806 (local address 127.19.238.126:37563)
0812 16:32:48.403910 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:48.403912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:48.403922 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:48.403952 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:48.403953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:48.403964 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:48.404043 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:48.404721 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.405273 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.406126 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.406399 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.406451 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:48.407939 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:32:48.407944 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:32:48.407944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:32:48.407956 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:32:48.409874 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:48.410352 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:48.410356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:48.410356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:48.410494 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:32:48.410844 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:32:48.410846 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:32:48.410846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:32:48.411114 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:32:48.411276 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 16:32:48.411395 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 16:32:48.411533 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:48.415418 30930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:48.410646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.67:52971)
0812 16:32:48.410704 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:48.410715 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:48.410758 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:48.410892 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:48.410893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:48.411027 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:48.411072 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:48.411074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.411713 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.411714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:48.412485 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:48.412488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.412989 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.412991 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:48.413055 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:48.413510 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:48.413514 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:48.415321 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 16:32:48.415490 30634 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:48.410716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:52971 (local address 127.19.238.126:37563)
0812 16:32:48.410753 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:48.410755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:48.410763 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:32:48.410818 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:48.410819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:48.410829 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:48.410877 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:48.411161 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.411628 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.412658 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.412916 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.412992 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:48.415422 (+ 2430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.67:52971: BlockingRecv error: recv got EOF from 127.19.238.67:52971 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:48 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:48.420099 30930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:48.415879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.67:57519)
0812 16:32:48.415976 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:48.415988 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:48.416039 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:48.416161 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:48.416162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:48.416271 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:48.416299 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:48.416300 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.417039 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.417040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:48.417553 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:48.417555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.417907 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.417908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:48.417944 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:48.418332 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:48.418336 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:48.420051 (+ 1715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":75}
W20260812 16:32:48.420240 30634 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:48.415966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:57519 (local address 127.19.238.126:37563)
0812 16:32:48.416018 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:48.416020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:48.416027 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:32:48.416081 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:48.416082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:48.416089 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:48.416143 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:48.416342 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.416948 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.417640 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:48.417828 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:48.417893 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:48.420149 (+ 2256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.67:57519: BlockingRecv error: recv got EOF from 127.19.238.67:57519 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260812 16:32:48.420272 30928 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 16:32:49 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:49 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:49.157648 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:49.152427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.65:42081)
0812 16:32:49.152577 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:49.152588 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:49.152644 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:49.152801 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:49.152801 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:49.152982 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:49.153034 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:49.153036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:49.153707 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:49.153708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:49.154257 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:49.154258 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:49.154670 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:49.154672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:49.154734 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:49.155277 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:49.155282 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:49.157543 (+ 2261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
W20260812 16:32:49.157719 30934 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:49.152527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:42081 (local address 127.19.238.126:37563)
0812 16:32:49.152668 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:49.152669 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:49.152678 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:32:49.152706 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:49.152707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:49.152717 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:49.152783 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:49.153106 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:49.153628 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:49.154338 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:49.154563 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:49.154622 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:49.157647 (+ 3025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.65:42081: BlockingRecv error: recv got EOF from 127.19.238.65:42081 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
Aug 12 16:32:49 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:49 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:49.294387 30935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:49.288793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.66:34135)
0812 16:32:49.289012 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:49.289026 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:49.289122 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:49.289275 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:49.289276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:49.289522 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:49.289607 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:49.289609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:49.290470 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:49.290471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:49.291044 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:49.291046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:49.291428 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:49.291429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:49.291484 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:49.292143 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:49.292148 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:49.294309 (+ 2161us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":130,"threads_started":1}
W20260812 16:32:49.294600 30934 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:49.288932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:34135 (local address 127.19.238.126:37563)
0812 16:32:49.288992 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:49.288995 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:49.289126 (+ 131us) server_negotiation.cc:407] Connection header received
0812 16:32:49.289169 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:49.289171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:49.289185 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:49.289253 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:49.289764 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:49.290380 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:49.291137 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:49.291335 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:49.291395 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:49.294521 (+ 3126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.66:34135: BlockingRecv error: recv got EOF from 127.19.238.66:34135 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 12 16:32:49 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:49 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:49.425788 30936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:49.420709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.67:58015)
0812 16:32:49.420865 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:49.420878 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:49.420944 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:49.421112 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:49.421113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:49.421292 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:49.421369 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:49.421433 (+ 64us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:49.422275 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:49.422276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:49.422847 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:49.422850 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:49.423244 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:49.423245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:49.423298 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:49.423874 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:49.423880 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:49.425718 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":89,"threads_started":1}
W20260812 16:32:49.425937 30934 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:49.420797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:58015 (local address 127.19.238.126:37563)
0812 16:32:49.420868 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:49.420870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:49.420944 (+ 74us) server_negotiation.cc:407] Connection header received
0812 16:32:49.421024 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:49.421025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:49.421039 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:49.421087 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:49.421523 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:49.422174 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:49.422945 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:49.423142 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:49.423196 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:49.425843 (+ 2647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.67:58015: BlockingRecv error: recv got EOF from 127.19.238.67:58015 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 16:32:50 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:50 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:50.163657 30937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:50.158278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.65:43623)
0812 16:32:50.158505 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:50.158516 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:50.158582 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:50.158762 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:50.158763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:50.158945 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:50.158997 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:50.158998 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:50.159768 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:50.159770 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:50.160401 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:50.160404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:50.160855 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:50.160856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:50.160912 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:50.161538 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:50.161543 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:50.163556 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":161,"threads_started":1}
W20260812 16:32:50.163743 30938 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:50.158483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:43623 (local address 127.19.238.126:37563)
0812 16:32:50.158618 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:50.158620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:50.158630 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:50.158657 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:50.158658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:50.158669 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:50.158748 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:50.159116 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:50.159659 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:50.160510 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:50.160723 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:50.160794 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:50.163659 (+ 2865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.65:43623: BlockingRecv error: recv got EOF from 127.19.238.65:43623 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
Aug 12 16:32:50 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:50 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:50.300731 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:50.295031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.66:54421)
0812 16:32:50.295183 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:50.295195 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:50.295268 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:50.295506 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:50.295507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:50.295704 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:50.295754 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:50.295756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:50.296721 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:50.296722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:50.297391 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:50.297393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:50.297862 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:50.297863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:50.297923 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:50.298443 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:50.298449 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:50.300633 (+ 2184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
W20260812 16:32:50.300796 30938 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:50.295195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:54421 (local address 127.19.238.126:37563)
0812 16:32:50.295289 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:50.295291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:50.295370 (+ 79us) server_negotiation.cc:407] Connection header received
0812 16:32:50.295403 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:50.295404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:50.295416 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:50.295482 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:50.295927 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:50.296630 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:50.297479 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:50.297765 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:50.297858 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:50.300735 (+ 2877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.66:54421: BlockingRecv error: recv got EOF from 127.19.238.66:54421 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 16:32:50 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:50 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:50.432113 30941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:50.426446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.67:52795)
0812 16:32:50.426648 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:50.426659 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:50.426713 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:50.426965 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:50.426966 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:50.427170 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:50.427226 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:50.427228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:50.428008 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:50.428010 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:50.428691 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:50.428694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:50.429072 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:50.429073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:50.429124 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:50.429726 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:50.429731 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:50.432047 (+ 2316us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":87,"threads_started":1}
W20260812 16:32:50.432212 30938 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:50.426644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:52795 (local address 127.19.238.126:37563)
0812 16:32:50.426749 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:50.426754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:50.426770 (+ 16us) server_negotiation.cc:407] Connection header received
0812 16:32:50.426808 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:50.426809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:50.426823 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:50.426919 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:50.427321 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:50.427920 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:50.428786 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:50.428990 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:50.429050 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:50.432144 (+ 3094us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.67:52795: BlockingRecv error: recv got EOF from 127.19.238.67:52795 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 12 16:32:51 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:51 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:51.169763 30942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:51.164395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.65:47509)
0812 16:32:51.164545 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:51.164557 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:51.164621 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:51.164830 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:51.164831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:51.165019 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:51.165088 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:51.165091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:51.165802 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:51.165803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:51.166465 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:51.166468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:51.166899 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:51.166900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:51.166951 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:51.167506 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:51.167512 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:51.169694 (+ 2182us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
W20260812 16:32:51.169965 30943 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:51.164473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:47509 (local address 127.19.238.126:37563)
0812 16:32:51.164641 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:51.164643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:51.164657 (+ 14us) server_negotiation.cc:407] Connection header received
0812 16:32:51.164681 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:51.164681 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:51.164692 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:51.164758 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:51.165175 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:51.165709 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:51.166579 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:51.166797 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:51.166872 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:51.169866 (+ 2994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.65:47509: BlockingRecv error: recv got EOF from 127.19.238.65:47509 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":109,"threads_started":1}
Aug 12 16:32:51 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:51 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:51.306288 30944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:51.301347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.66:34947)
0812 16:32:51.301528 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:51.301539 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:51.301617 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:51.301773 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:51.301774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:51.302021 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:51.302115 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:51.302118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:51.302812 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:51.302813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:51.303393 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:51.303395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:51.303769 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:51.303770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:51.303853 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:51.304321 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:51.304326 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:51.306222 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":123,"threads_started":1}
W20260812 16:32:51.306452 30943 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:51.301417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:34947 (local address 127.19.238.126:37563)
0812 16:32:51.301494 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:51.301497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:51.301642 (+ 145us) server_negotiation.cc:407] Connection header received
0812 16:32:51.301678 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:51.301679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:51.301691 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:51.301754 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:51.302205 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:51.302720 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:51.303484 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:51.303690 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:51.303770 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:51.306364 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.66:34947: BlockingRecv error: recv got EOF from 127.19.238.66:34947 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 16:32:51 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:51 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:51.437932 30945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:51.432689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.67:45061)
0812 16:32:51.432938 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:51.432953 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:51.433066 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:51.433212 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:51.433213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:51.433470 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:51.433532 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:51.433535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:51.434287 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:51.434288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:51.434838 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:51.434840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:51.435223 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:51.435224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:51.435285 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:51.435863 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:51.435869 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:51.437832 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":124,"threads_started":1}
W20260812 16:32:51.438000 30943 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:51.432817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:45061 (local address 127.19.238.126:37563)
0812 16:32:51.432922 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:51.432924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:51.433045 (+ 121us) server_negotiation.cc:407] Connection header received
0812 16:32:51.433130 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:51.433132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:51.433148 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:51.433206 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:51.433643 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:51.434187 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:51.434936 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:51.435140 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:51.435222 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:51.437934 (+ 2712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.67:45061: BlockingRecv error: recv got EOF from 127.19.238.67:45061 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 12 16:32:52 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:52 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:52.180450 30946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:52.170364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.65:33853)
0812 16:32:52.170550 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:52.170561 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:52.170626 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:52.170782 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:52.170782 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:52.170963 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:52.171017 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:52.171019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:52.171702 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:52.171702 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:52.172314 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:52.172317 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:52.172724 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:52.172725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:52.172775 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:52.178363 (+ 5588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:52.178370 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:52.180376 (+ 2006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":109,"threads_started":1}
W20260812 16:32:52.180656 30947 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:52.170485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:33853 (local address 127.19.238.126:37563)
0812 16:32:52.170642 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:52.170644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:52.170654 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:52.170680 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:52.170681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:52.170691 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:52.170758 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:52.171096 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:52.171619 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:52.172430 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:52.172633 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:52.172692 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:52.180582 (+ 7890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.65:33853: BlockingRecv error: recv got EOF from 127.19.238.65:33853 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
Aug 12 16:32:52 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:52 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:52.311990 30948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:52.306860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.66:33091)
0812 16:32:52.307086 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:52.307107 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:52.307214 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:52.307397 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:52.307398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:52.307615 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:52.307663 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:52.307665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:52.308456 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:52.308458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:52.309004 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:52.309006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:52.309390 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:52.309392 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:52.309457 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:52.309983 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:52.309991 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:52.311908 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":96,"threads_started":1}
W20260812 16:32:52.312098 30947 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:52.306959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:33091 (local address 127.19.238.126:37563)
0812 16:32:52.307095 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:52.307097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:52.307189 (+ 92us) server_negotiation.cc:407] Connection header received
0812 16:32:52.307297 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:52.307298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:52.307310 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:52.307392 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:52.307755 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:52.308359 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:52.309096 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:52.309309 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:52.309391 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:52.312017 (+ 2626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.66:33091: BlockingRecv error: recv got EOF from 127.19.238.66:33091 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 12 16:32:52 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:52 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:52.443729 30949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:52.438735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.67:43171)
0812 16:32:52.438921 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:52.438935 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:52.439005 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:52.439218 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:52.439218 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:52.439418 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:52.439467 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:52.439469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:52.440229 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:52.440230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:52.440808 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:52.440811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:52.441209 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:52.441210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:52.441251 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:52.441732 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:52.441737 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:52.443667 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":125,"threads_started":1}
W20260812 16:32:52.443935 30947 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:52.438909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:43171 (local address 127.19.238.126:37563)
0812 16:32:52.438986 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:52.438988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:52.438999 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:32:52.439093 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:52.439094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:52.439106 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:52.439173 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:52.439549 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:52.440138 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:52.440922 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:52.441131 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:52.441198 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:52.443806 (+ 2608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.67:43171: BlockingRecv error: recv got EOF from 127.19.238.67:43171 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 16:32:53 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:53 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:53.186440 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:53.181016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.65:58323)
0812 16:32:53.181181 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:53.181193 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:53.181258 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:53.181456 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:53.181457 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:53.181638 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:53.181691 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:53.181692 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:53.182442 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:53.182443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:53.183144 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:53.183147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:53.183514 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:53.183516 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:53.183569 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:53.184151 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:53.184157 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:53.186379 (+ 2222us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":103,"threads_started":1}
W20260812 16:32:53.186614 30951 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:53.181142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:58323 (local address 127.19.238.126:37563)
0812 16:32:53.181279 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:53.181281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:53.181290 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:32:53.181319 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:53.181320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:53.181333 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:53.181405 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:53.181790 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:53.182305 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:53.183219 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:53.183421 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:53.183481 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:53.186501 (+ 3020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.65:58323: BlockingRecv error: recv got EOF from 127.19.238.65:58323 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
Aug 12 16:32:53 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:53 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:53.317898 30953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:53.312603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.66:34615)
0812 16:32:53.312779 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:53.312810 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:53.312897 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:53.313092 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:53.313093 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:53.313289 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:53.313338 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:53.313340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:53.314068 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:53.314070 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:53.314626 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:53.314629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:53.315038 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:53.315040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:53.315098 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:53.315684 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:53.315689 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:53.317782 (+ 2093us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":68,"threads_started":1}
W20260812 16:32:53.317963 30951 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:53.312770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:34615 (local address 127.19.238.126:37563)
0812 16:32:53.312844 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:53.312847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:53.312872 (+ 25us) server_negotiation.cc:407] Connection header received
0812 16:32:53.312992 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:53.312993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:53.313013 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:53.313084 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:53.313432 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:53.313969 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:53.314728 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:53.314928 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:53.315040 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:53.317886 (+ 2846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.66:34615: BlockingRecv error: recv got EOF from 127.19.238.66:34615 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 16:32:53 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:53 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:53.449716 30954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:53.444374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.67:55235)
0812 16:32:53.444613 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:53.444626 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:53.444692 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:53.444827 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:53.444827 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:53.445013 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:53.445064 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:53.445066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:53.445834 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:53.445836 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:53.446448 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:53.446450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:53.446891 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:53.446892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:53.446957 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:53.447434 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:53.447440 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:53.449604 (+ 2164us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":159,"threads_started":1}
W20260812 16:32:53.449776 30951 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:53.444464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:55235 (local address 127.19.238.126:37563)
0812 16:32:53.444556 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:53.444558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:53.444692 (+ 134us) server_negotiation.cc:407] Connection header received
0812 16:32:53.444743 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:53.444744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:53.444757 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:53.444818 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:53.445156 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:53.445722 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:53.446543 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:53.446767 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:53.446832 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:53.449703 (+ 2871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.67:55235: BlockingRecv error: recv got EOF from 127.19.238.67:55235 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 16:32:54 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:54 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:54.192456 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:54.187039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.65:60423)
0812 16:32:54.187255 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:54.187267 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:54.187341 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:54.187506 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:54.187506 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:54.187678 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:54.187729 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:54.187731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:54.188568 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:54.188569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:54.189094 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:54.189095 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:54.189542 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:54.189543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:54.189600 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:54.190224 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:54.190231 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:54.192386 (+ 2155us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":115,"threads_started":1}
W20260812 16:32:54.192656 30956 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:54.187138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:60423 (local address 127.19.238.126:37563)
0812 16:32:54.187314 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:54.187316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:54.187364 (+ 48us) server_negotiation.cc:407] Connection header received
0812 16:32:54.187395 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:54.187395 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:54.187407 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:54.187480 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:54.187799 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:54.188481 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:54.189189 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:54.189415 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:54.189491 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:54.192578 (+ 3087us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.65:60423: BlockingRecv error: recv got EOF from 127.19.238.65:60423 (error 108)
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":89,"threads_started":1}
Aug 12 16:32:54 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:54 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:54.324760 30957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:54.318476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.66:49995)
0812 16:32:54.318690 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:54.318702 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:54.318786 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:54.318941 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:54.318942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:54.319173 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:54.319238 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:54.319241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:54.320063 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:54.320065 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:54.321154 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:54.321159 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:54.321597 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:54.321598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:54.321672 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:54.322195 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:54.322202 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:54.324650 (+ 2448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":93,"threads_started":1}
W20260812 16:32:54.324826 30956 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:54.318582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:49995 (local address 127.19.238.126:37563)
0812 16:32:54.318644 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:54.318647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:54.318806 (+ 159us) server_negotiation.cc:407] Connection header received
0812 16:32:54.318860 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:54.318860 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:54.318874 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:54.318915 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:54.319358 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:54.319973 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:54.321269 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:54.321508 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:54.321574 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:54.324758 (+ 3184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.66:49995: BlockingRecv error: recv got EOF from 127.19.238.66:49995 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 12 16:32:54 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:54 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:54.455065 30959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:54.450293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.67:52645)
0812 16:32:54.450455 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:54.450466 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:54.450534 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:54.450649 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:54.450649 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:54.450821 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:54.450871 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:54.450873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:54.451532 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:54.451533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:54.452071 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:54.452073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:54.452428 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:54.452429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:54.452468 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:54.453010 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:54.453015 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:54.454972 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
W20260812 16:32:54.455221 30956 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:54.450364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:52645 (local address 127.19.238.126:37563)
0812 16:32:54.450420 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:54.450422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:54.450533 (+ 111us) server_negotiation.cc:407] Connection header received
0812 16:32:54.450579 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:54.450580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:54.450592 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:54.450643 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:54.450927 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:54.451430 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:54.452162 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:54.452354 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:54.452417 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:54.455116 (+ 2699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.67:52645: BlockingRecv error: recv got EOF from 127.19.238.67:52645 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 12 16:32:55 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:55 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:55.198117 30960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:55.193000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.65:41829)
0812 16:32:55.193182 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:55.193194 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:55.193262 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:55.193398 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:55.193399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:55.193626 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:55.193680 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:55.193683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:55.194380 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:55.194382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:55.194949 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:55.194951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:55.195420 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:55.195421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:55.195500 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:55.196072 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:55.196077 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:55.198053 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":102,"threads_started":1}
W20260812 16:32:55.198334 30961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:55.193113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:41829 (local address 127.19.238.126:37563)
0812 16:32:55.193241 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:55.193243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:55.193253 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:55.193315 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:55.193316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:55.193326 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:55.193392 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:55.193761 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:55.194280 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:55.195076 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:55.195350 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:55.195420 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:55.198250 (+ 2830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.65:41829: BlockingRecv error: recv got EOF from 127.19.238.65:41829 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
Aug 12 16:32:55 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:55 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:55.331094 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:55.325442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.66:38733)
0812 16:32:55.325659 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:55.325674 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:55.325782 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:55.325993 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:55.325994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:55.326197 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:55.326250 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:55.326252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:55.327055 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:55.327056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:55.327814 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:55.327817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:55.328442 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:55.328443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:55.328521 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:55.329033 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:55.329038 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:55.331020 (+ 1982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":98,"threads_started":1}
W20260812 16:32:55.331223 30961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:55.325562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:38733 (local address 127.19.238.126:37563)
0812 16:32:55.325662 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:55.325665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:55.325759 (+ 94us) server_negotiation.cc:407] Connection header received
0812 16:32:55.325878 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:55.325879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:55.325895 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:55.325970 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:55.326372 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:55.326944 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:55.327928 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:55.328225 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:55.328302 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:55.331138 (+ 2836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.66:38733: BlockingRecv error: recv got EOF from 127.19.238.66:38733 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 16:32:55 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:55 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:55.462792 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:55.455739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.67:43631)
0812 16:32:55.455905 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:55.455921 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:55.456005 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:55.456220 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:55.456221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:55.456456 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:55.456526 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:55.456528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:55.457677 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:55.457678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:55.458671 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:55.458675 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:55.459260 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:55.459261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:55.459333 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:55.460052 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:55.460058 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:55.462682 (+ 2624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":111,"threads_started":1}
W20260812 16:32:55.462913 30961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:55.455871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:43631 (local address 127.19.238.126:37563)
0812 16:32:55.455928 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:55.455930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:55.456014 (+ 84us) server_negotiation.cc:407] Connection header received
0812 16:32:55.456094 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:55.456095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:55.456114 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:55.456212 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:55.456630 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:55.457558 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:55.458795 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:55.459164 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:55.459253 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:55.462831 (+ 3578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.67:43631: BlockingRecv error: recv got EOF from 127.19.238.67:43631 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 12 16:32:56 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:56 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:56.203776 30965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:56.198687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.65:48953)
0812 16:32:56.198840 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:56.198888 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:56.198958 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:56.199215 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:56.199217 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:56.199419 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:56.199474 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:56.199476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:56.200224 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:56.200225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:56.200780 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:56.200782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:56.201155 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:56.201156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:56.201197 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:56.201663 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:56.201668 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:56.203678 (+ 2010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":63,"threads_started":1}
W20260812 16:32:56.203995 30966 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:56.198848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:48953 (local address 127.19.238.126:37563)
0812 16:32:56.199046 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:56.199048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:56.199059 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:32:56.199086 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:56.199087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:56.199106 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:56.199188 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:56.199579 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:56.200149 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:56.200866 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:56.201086 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:56.201144 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:56.203817 (+ 2673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.65:48953: BlockingRecv error: recv got EOF from 127.19.238.65:48953 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":110,"threads_started":1}
Aug 12 16:32:56 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:56 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:56.336870 30967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:56.331651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.66:39707)
0812 16:32:56.331873 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:56.331888 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:56.331997 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:56.332182 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:56.332183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:56.332385 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:56.332436 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:56.332438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:56.333364 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:56.333366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:56.333934 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:56.333937 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:56.334312 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:56.334313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:56.334372 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:56.334899 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:56.334904 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:56.336764 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":95,"threads_started":1}
W20260812 16:32:56.336923 30966 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:56.331740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:39707 (local address 127.19.238.126:37563)
0812 16:32:56.331892 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:56.331894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:56.331975 (+ 81us) server_negotiation.cc:407] Connection header received
0812 16:32:56.332082 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:56.332083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:56.332096 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:56.332173 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:56.332526 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:56.333218 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:56.334024 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:56.334226 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:56.334292 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:56.336857 (+ 2565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.66:39707: BlockingRecv error: recv got EOF from 127.19.238.66:39707 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 12 16:32:56 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:56 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:56.469043 30968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:56.463430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.67:46377)
0812 16:32:56.463619 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:56.463630 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:56.463691 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:56.463865 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:56.463866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:56.464048 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:56.464101 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:56.464103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:56.464820 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:56.464821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:56.465446 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:56.465449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:56.465870 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:56.465872 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:56.465924 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:56.466510 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:56.466515 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:56.468932 (+ 2417us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":116,"threads_started":1}
W20260812 16:32:56.469111 30966 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:56.463556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:46377 (local address 127.19.238.126:37563)
0812 16:32:56.463628 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:56.463631 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:56.463689 (+ 58us) server_negotiation.cc:407] Connection header received
0812 16:32:56.463738 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:56.463739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:56.463751 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:56.463814 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:56.464177 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:56.464720 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:56.465568 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:56.465774 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:56.465847 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:56.469035 (+ 3188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.67:46377: BlockingRecv error: recv got EOF from 127.19.238.67:46377 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 16:32:57 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:57 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:57.213711 30969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:57.204383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.65:37327)
0812 16:32:57.204615 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:57.204632 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:57.204723 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:57.204969 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:57.204970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:57.205154 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:57.205210 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:57.205212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:57.206023 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:57.206024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:57.206625 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:57.206627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:57.207090 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:57.207091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:57.207152 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:57.211162 (+ 4010us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:57.211169 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:57.213629 (+ 2460us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":88,"threads_started":1}
W20260812 16:32:57.213898 30970 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:57.204593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:37327 (local address 127.19.238.126:37563)
0812 16:32:57.204774 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:57.204776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:57.204786 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:32:57.204813 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:57.204814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:57.204826 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:57.204902 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:57.205345 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:57.205907 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:57.206737 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:57.206978 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:57.207053 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:57.213787 (+ 6734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.65:37327: BlockingRecv error: recv got EOF from 127.19.238.65:37327 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":78,"threads_started":1}
Aug 12 16:32:57 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:57 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:57.342633 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:57.337573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.66:35199)
0812 16:32:57.337736 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:57.337747 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:57.337803 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:57.337956 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:57.337957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:57.338137 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:57.338190 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:57.338192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:57.338943 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:57.338944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:57.339482 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:57.339483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:57.339941 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:57.339942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:57.339992 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:57.340513 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:57.340518 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:57.342570 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
W20260812 16:32:57.342813 30970 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:57.337686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:35199 (local address 127.19.238.126:37563)
0812 16:32:57.337792 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:57.337794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:57.337807 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:32:57.337857 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:57.337858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:57.337869 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:57.337941 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:57.338271 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:57.338853 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:57.339593 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:57.339802 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:57.339954 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:57.342704 (+ 2750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.66:35199: BlockingRecv error: recv got EOF from 127.19.238.66:35199 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 16:32:57 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:57 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:57.474766 30972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:57.469790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.67:60513)
0812 16:32:57.469958 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:57.469969 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:57.470037 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:57.470161 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:57.470162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:57.470332 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:57.470380 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:57.470382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:57.471112 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:57.471113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:57.471691 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:57.471693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:57.472112 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:57.472113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:57.472160 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:57.472643 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:57.472648 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:57.474686 (+ 2038us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":117,"threads_started":1}
W20260812 16:32:57.474915 30970 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:57.469880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:60513 (local address 127.19.238.126:37563)
0812 16:32:57.469948 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:57.469950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:57.470047 (+ 97us) server_negotiation.cc:407] Connection header received
0812 16:32:57.470076 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:57.470076 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:57.470091 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:57.470150 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:57.470484 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:57.471009 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:57.471775 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:57.472040 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:57.472106 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:57.474829 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.67:60513: BlockingRecv error: recv got EOF from 127.19.238.67:60513 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 12 16:32:58 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:58 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:58.220321 30973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:58.214447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.65:54649)
0812 16:32:58.214655 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:58.214667 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:58.214738 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:58.214932 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:58.214933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:58.215137 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:58.215190 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:58.215193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:58.216134 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:58.216136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:58.216771 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:58.216773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:58.217228 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:58.217230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:58.217288 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:58.218013 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:58.218018 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:58.220254 (+ 2236us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":105,"threads_started":1}
W20260812 16:32:58.220438 30974 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:58.214594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:54649 (local address 127.19.238.126:37563)
0812 16:32:58.214754 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:58.214756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:58.214767 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:32:58.214798 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:58.214799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:58.214811 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:58.214902 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:58.215306 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:58.216045 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:58.216892 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:58.217133 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:58.217198 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:58.220379 (+ 3181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.65:54649: BlockingRecv error: recv got EOF from 127.19.238.65:54649 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
Aug 12 16:32:58 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:58 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:58.348244 30975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:58.343245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.66:40535)
0812 16:32:58.343428 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:58.343447 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:58.343515 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:58.343698 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:58.343699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:58.343927 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:58.343981 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:58.343983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:58.344687 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:58.344688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:58.345237 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:58.345238 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:58.345595 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:58.345596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:58.345645 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:58.346199 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:58.346205 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:58.348170 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":122,"threads_started":1}
W20260812 16:32:58.348358 30974 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:58.343339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:40535 (local address 127.19.238.126:37563)
0812 16:32:58.343392 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:58.343395 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:58.343494 (+ 99us) server_negotiation.cc:407] Connection header received
0812 16:32:58.343588 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:58.343588 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:58.343599 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:58.343643 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:58.344079 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:58.344601 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:58.345316 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:58.345512 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:58.345563 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:58.348285 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.66:40535: BlockingRecv error: recv got EOF from 127.19.238.66:40535 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 12 16:32:58 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:58 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:58.480278 30976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:58.475323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.67:39593)
0812 16:32:58.475577 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:58.475594 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:58.475683 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:58.475866 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:58.475867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:58.476065 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:58.476114 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:58.476116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:58.476815 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:58.476817 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:58.477345 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:58.477347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:58.477722 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:58.477723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:58.477786 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:58.478253 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:58.478257 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:58.480214 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":82,"threads_started":1}
W20260812 16:32:58.480432 30974 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:58.475532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:39593 (local address 127.19.238.126:37563)
0812 16:32:58.475606 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:58.475608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:58.475677 (+ 69us) server_negotiation.cc:407] Connection header received
0812 16:32:58.475757 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:58.475758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:58.475779 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:58.475875 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:58.476207 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:58.476716 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:58.477451 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:58.477643 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:58.477723 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:58.480343 (+ 2620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.67:39593: BlockingRecv error: recv got EOF from 127.19.238.67:39593 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 16:32:59 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:59 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:59.225819 30977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:59.220892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.65:58947)
0812 16:32:59.221061 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:59.221074 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:59.221157 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:59.221295 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:59.221296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:59.221512 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:59.221562 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:59.221564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:59.222227 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:59.222228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:59.222786 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:59.222788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:59.223214 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:59.223216 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:59.223278 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:59.223756 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:59.223760 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:59.225745 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":84,"threads_started":1}
W20260812 16:32:59.225975 30978 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:59.221007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:58947 (local address 127.19.238.126:37563)
0812 16:32:59.221158 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:59.221160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:59.221169 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:32:59.221209 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:59.221209 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:59.221220 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:59.221270 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:59.221635 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:59.222154 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:59.222872 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:59.223121 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:59.223180 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:59.225879 (+ 2699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.65:58947: BlockingRecv error: recv got EOF from 127.19.238.65:58947 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":75,"threads_started":1}
Aug 12 16:32:59 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:59 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:59.354321 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:59.348902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.66:51315)
0812 16:32:59.349103 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:59.349114 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:59.349185 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:59.349433 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:59.349434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:59.349617 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:59.349665 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:59.349666 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:59.350429 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:59.350430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:59.350990 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:59.350991 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:59.351337 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:59.351338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:59.351431 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:59.352120 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:59.352126 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:59.354207 (+ 2081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":94,"threads_started":1}
W20260812 16:32:59.354360 30978 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:59.349041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:51315 (local address 127.19.238.126:37563)
0812 16:32:59.349112 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:59.349114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:59.349174 (+ 60us) server_negotiation.cc:407] Connection header received
0812 16:32:59.349270 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:59.349272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:59.349286 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:59.349359 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:59.349740 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:59.350288 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:59.351073 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:59.351261 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:59.351316 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:59.354295 (+ 2979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.66:51315: BlockingRecv error: recv got EOF from 127.19.238.66:51315 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 16:32:59 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:32:59 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:32:59.485953 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:32:59.480779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.67:40911)
0812 16:32:59.480981 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 16:32:59.480994 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:32:59.481083 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:32:59.481260 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:32:59.481262 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:32:59.481481 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:32:59.481531 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:59.481534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:59.482199 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:59.482200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:59.482780 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:32:59.482782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:32:59.483295 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:32:59.483297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:32:59.483359 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:59.483943 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:32:59.483949 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:32:59.485881 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":119,"threads_started":1}
W20260812 16:32:59.486100 30978 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:32:59.480885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:40911 (local address 127.19.238.126:37563)
0812 16:32:59.480944 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 16:32:59.480946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:32:59.481139 (+ 193us) server_negotiation.cc:407] Connection header received
0812 16:32:59.481172 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:32:59.481173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:32:59.481187 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:32:59.481235 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:32:59.481610 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:59.482119 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:59.482877 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:32:59.483167 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:32:59.483273 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:32:59.485999 (+ 2726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.67:40911: BlockingRecv error: recv got EOF from 127.19.238.67:40911 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 12 16:33:00 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:33:00 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:33:00.232297 30981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:00.226475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.65:32955)
0812 16:33:00.226684 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:00.226696 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:00.226768 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:00.226959 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:00.226960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:00.227166 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:00.227227 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:00.227229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:00.228046 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:00.228047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:00.228673 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:00.228677 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:00.229122 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:00.229124 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:00.229196 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:00.229948 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:33:00.229954 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:00.232207 (+ 2253us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":107,"threads_started":1}
W20260812 16:33:00.232407 30982 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:33:00.226617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:32955 (local address 127.19.238.126:37563)
0812 16:33:00.226739 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:00.226741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:00.226752 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:33:00.226857 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:00.226858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:00.226869 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:00.226933 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:00.227323 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:00.227935 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:00.228796 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:00.229028 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:00.229120 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:00.232330 (+ 3210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.65:32955: BlockingRecv error: recv got EOF from 127.19.238.65:32955 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
Aug 12 16:33:00 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:33:00 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:33:00.360464 30983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:00.355148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.66:39419)
0812 16:33:00.355395 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:00.355407 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:00.355476 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:00.355618 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:00.355618 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:00.355796 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:00.355876 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:00.355878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:00.356540 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:00.356541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:00.357106 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:00.357109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:00.357527 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:00.357528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:00.357575 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:00.358167 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:33:00.358172 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:00.360387 (+ 2215us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":85,"threads_started":1}
W20260812 16:33:00.360625 30982 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:33:00.355277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:39419 (local address 127.19.238.126:37563)
0812 16:33:00.355355 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:00.355357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:00.355471 (+ 114us) server_negotiation.cc:407] Connection header received
0812 16:33:00.355542 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:00.355543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:00.355556 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:00.355605 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:00.355947 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:00.356448 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:00.357223 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:00.357442 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:00.357512 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:00.360539 (+ 3027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.66:39419: BlockingRecv error: recv got EOF from 127.19.238.66:39419 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 16:33:00 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:33:00 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:33:00.491355 30984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:00.486465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.67:36481)
0812 16:33:00.486688 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:00.486701 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:00.486794 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:00.486979 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:00.486980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:00.487175 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:00.487221 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:00.487223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:00.487937 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:00.487939 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:00.488514 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:00.488516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:00.488952 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:00.488953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:00.489018 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:00.489488 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:33:00.489493 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:00.491254 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":108,"threads_started":1}
W20260812 16:33:00.491407 30982 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:33:00.486611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:36481 (local address 127.19.238.126:37563)
0812 16:33:00.486688 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:00.486690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:00.486773 (+ 83us) server_negotiation.cc:407] Connection header received
0812 16:33:00.486877 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:00.486879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:00.486894 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:00.486970 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:00.487306 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:00.487805 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:00.488622 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:00.488861 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:00.488949 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:00.491339 (+ 2390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.67:36481: BlockingRecv error: recv got EOF from 127.19.238.67:36481 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 16:33:01 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:33:01 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:33:01.237990 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:01.232946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.65:47719)
0812 16:33:01.233103 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:01.233115 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:01.233172 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:01.233323 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:01.233324 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:01.233544 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:01.233596 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:01.233599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:01.234325 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:01.234327 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:01.235006 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:01.235009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:01.235435 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:01.235436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:01.235481 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:01.236038 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:33:01.236043 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:01.237922 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
W20260812 16:33:01.238162 30986 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:33:01.233035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:47719 (local address 127.19.238.126:37563)
0812 16:33:01.233174 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:01.233176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:01.233185 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:33:01.233223 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:01.233224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:01.233234 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:01.233300 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:01.233680 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:01.234216 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:01.235097 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:01.235313 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:01.235373 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:01.238071 (+ 2698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.65:47719: BlockingRecv error: recv got EOF from 127.19.238.65:47719 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
Aug 12 16:33:01 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:33:01 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:33:01.366187 30987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:01.361074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.66:58535)
0812 16:33:01.361230 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:01.361243 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:01.361334 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:01.361634 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:01.361635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:01.361821 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:01.361872 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:01.361874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:01.362556 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:01.362557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:01.363133 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:01.363135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:01.363538 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:01.363539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:01.363589 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:01.364089 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:33:01.364095 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:01.366121 (+ 2026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":96,"threads_started":1}
W20260812 16:33:01.366387 30986 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:33:01.361387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:58535 (local address 127.19.238.126:37563)
0812 16:33:01.361490 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:01.361492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:01.361508 (+ 16us) server_negotiation.cc:407] Connection header received
0812 16:33:01.361538 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:01.361539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:01.361553 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:01.361620 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:01.361948 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:01.362462 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:01.363214 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:01.363429 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:01.363488 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:01.366296 (+ 2808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.66:58535: BlockingRecv error: recv got EOF from 127.19.238.66:58535 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 12 16:33:01 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:33:01 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:33:01.497985 30988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:01.492155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.67:59085)
0812 16:33:01.492438 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:01.492452 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:01.492561 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:01.492762 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:01.492764 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:01.492977 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:01.493033 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:01.493036 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:01.493888 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:01.493890 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:01.494764 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:01.494768 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:01.495148 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:01.495150 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:01.495200 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:01.495690 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:33:01.495695 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:01.497919 (+ 2224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":131,"threads_started":1}
W20260812 16:33:01.498107 30986 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:33:01.492294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:59085 (local address 127.19.238.126:37563)
0812 16:33:01.492436 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:01.492439 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:01.492536 (+ 97us) server_negotiation.cc:407] Connection header received
0812 16:33:01.492652 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:01.492653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:01.492669 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:01.492754 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:01.493141 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:01.493765 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:01.494842 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:01.495056 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:01.495120 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:01.498030 (+ 2910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.67:59085: BlockingRecv error: recv got EOF from 127.19.238.67:59085 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 12 16:33:02 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:33:02 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:33:02.248814 30989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:02.238502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.65:47623)
0812 16:33:02.238689 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:02.238701 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:02.238785 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:02.239018 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:02.239019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:02.239259 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:02.239311 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:02.239313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:02.240099 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:02.240100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:02.240621 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:02.240622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:02.240965 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:02.240966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:02.241025 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:02.246710 (+ 5685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:33:02.246716 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:02.248726 (+ 2010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":87,"threads_started":1}
W20260812 16:33:02.248983 30990 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:33:02.238620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:47623 (local address 127.19.238.126:37563)
0812 16:33:02.238754 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:02.238755 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:02.238765 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:33:02.238849 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:02.238850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:02.238863 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:02.238947 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:02.239394 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:02.240024 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:02.240692 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:02.240900 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:02.240951 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:02.248877 (+ 7926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.65:47623: BlockingRecv error: recv got EOF from 127.19.238.65:47623 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":65,"threads_started":1}
Aug 12 16:33:02 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:33:02 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:33:02.371809 30991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:02.366797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.66:47817)
0812 16:33:02.366972 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:02.366987 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:02.367065 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:02.367239 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:02.367240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:02.367437 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:02.367486 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:02.367496 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:02.368256 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:02.368258 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:02.368836 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:02.368838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:02.369233 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:02.369234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:02.369300 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:02.369760 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:33:02.369764 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:02.371734 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":106,"threads_started":1}
W20260812 16:33:02.371992 30990 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:33:02.366924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:47817 (local address 127.19.238.126:37563)
0812 16:33:02.367045 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:02.367047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:02.367058 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:33:02.367133 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:02.367134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:02.367147 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:02.367231 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:02.367601 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:02.368173 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:02.368933 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:02.369142 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:02.369230 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:02.371904 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.66:47817: BlockingRecv error: recv got EOF from 127.19.238.66:47817 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 12 16:33:02 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:33:02 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.67@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:33:02.503687 30992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:02.498697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.67:49093)
0812 16:33:02.498879 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:02.498891 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:02.498968 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:02.499129 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:02.499131 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:02.499381 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:02.499439 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:02.499441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:02.500228 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:02.500229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:02.500779 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:02.500781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:02.501144 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:02.501145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:02.501185 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:02.501648 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:33:02.501653 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:02.503591 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":99,"threads_started":1}
W20260812 16:33:02.504022 30990 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:33:02.498786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:49093 (local address 127.19.238.126:37563)
0812 16:33:02.498866 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:02.498869 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:02.498949 (+ 80us) server_negotiation.cc:407] Connection header received
0812 16:33:02.499043 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:02.499044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:02.499062 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:02.499115 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:02.499513 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:02.500151 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:02.500883 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:02.501073 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:02.501133 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:02.503967 (+ 2834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.67:49093: BlockingRecv error: recv got EOF from 127.19.238.67:49093 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 16:33:03 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:33:03 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.65@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:33:03.254405 30993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:03.249474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.65:52131)
0812 16:33:03.249633 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:03.249644 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:03.249701 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:03.249877 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:03.249878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:03.250048 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:03.250100 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:03.250102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:03.250936 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:03.250937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:03.251546 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:03.251549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:03.251923 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:03.251923 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:03.251965 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:03.252519 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:33:03.252524 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:03.254342 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
W20260812 16:33:03.254560 30994 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:33:03.249583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:52131 (local address 127.19.238.126:37563)
0812 16:33:03.249751 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:03.249753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:03.249762 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:33:03.249794 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:03.249795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:03.249805 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:03.249871 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:03.250195 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:03.250865 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:03.251625 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:03.251819 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:03.251918 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:03.254469 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.65:52131: BlockingRecv error: recv got EOF from 127.19.238.65:52131 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":120,"threads_started":1}
Aug 12 16:33:03 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:33:03 dist-test-slave-mvvj krb5kdc[29793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.238.66@KRBTEST.COM for oryx/127.19.238.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:33:03.377408 30995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:03.372483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:37563 (local address 127.19.238.66:37945)
0812 16:33:03.372667 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:03.372679 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:03.372745 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:03.372904 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:03.372904 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:03.373087 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:03.373141 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:03.373143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:03.373898 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:03.373899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:03.374461 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:03.374463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:03.374840 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:03.374841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:03.374883 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:03.375371 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:33:03.375376 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:03.377340 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.238.126:37563: Server oryx/127.19.238.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":125,"threads_started":1}
W20260812 16:33:03.377530 30994 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:33:03.372595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:37945 (local address 127.19.238.126:37563)
0812 16:33:03.372724 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:03.372726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:03.372737 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:33:03.372822 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:03.372823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:03.372834 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:03.372884 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:03.373215 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:03.373758 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:03.374556 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:03.374767 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:03.374829 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:03.377453 (+ 2624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.238.66:37945: BlockingRecv error: recv got EOF from 127.19.238.66:37945 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
I20260812 16:33:03.412334 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 30464
I20260812 16:33:03.419750 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 30635
I20260812 16:33:03.427134 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 30781
I20260812 16:33:03.433467 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 30382
2026-08-12T16:33:03Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17750 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T16:33:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:33:03Z Disabled control of system clock
I20260812 16:33:03.469012 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:45297
--webserver_interface=127.19.238.126
--webserver_port=0
--builtin_ntp_servers=127.19.238.84:46653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:45297
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 16:33:03.558545 30999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:33:03.558772 30999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:33:03.558794 30999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:33:03.560053 30999 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:33:03.560101 30999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:33:03.560113 30999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:33:03.560127 30999 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:33:03.560138 30999 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:33:03.561580 30999 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:46653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:45297
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:45297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:33:03.561872 30999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:33:03.562207 30999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:33:03.567435 31006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:03.567451 31009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:03.567483 31005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:33:03.567770 30999 server_base.cc:1034] running on GCE node
I20260812 16:33:03.568096 30999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:33:03.568428 30999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:33:03.569621 30999 hybrid_clock.cc:648] HybridClock initialized: now 1786552383569602 us; error 28 us; skew 500 ppm
I20260812 16:33:03.571015 30999 webserver.cc:460] Webserver started at http://127.19.238.126:39449/ using document root <none> and password file <none>
I20260812 16:33:03.571295 30999 fs_manager.cc:359] Metadata directory not provided
I20260812 16:33:03.571374 30999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:33:03.571549 30999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:33:03.572865 30999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "7fb172205a1c4f6e9daaa50df34de8ab"
format_stamp: "Formatted at 2026-08-12 16:33:03 on dist-test-slave-mvvj"
server_key: "d62cfa782d707b458b37d501b3b62820"
server_key_iv: "a70d38f74bbc262fe72b20721591627f"
server_key_version: "encryptionkey@0"
I20260812 16:33:03.573290 30999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "7fb172205a1c4f6e9daaa50df34de8ab"
format_stamp: "Formatted at 2026-08-12 16:33:03 on dist-test-slave-mvvj"
server_key: "d62cfa782d707b458b37d501b3b62820"
server_key_iv: "a70d38f74bbc262fe72b20721591627f"
server_key_version: "encryptionkey@0"
I20260812 16:33:03.574832 30999 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:33:03.575668 31017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:03.575917 30999 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:33:03.576072 30999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "7fb172205a1c4f6e9daaa50df34de8ab"
format_stamp: "Formatted at 2026-08-12 16:33:03 on dist-test-slave-mvvj"
server_key: "d62cfa782d707b458b37d501b3b62820"
server_key_iv: "a70d38f74bbc262fe72b20721591627f"
server_key_version: "encryptionkey@0"
I20260812 16:33:03.576160 30999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:33:03.589046 30999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:33:03.589442 30999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:33:03.589627 30999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:33:03.594148 30999 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:45297
I20260812 16:33:03.594204 31075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:45297 every 8 connection(s)
I20260812 16:33:03.594622 30999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:33:03.595395 31076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:33:03.598215 31076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab: Bootstrap starting.
I20260812 16:33:03.598860 31076 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab: Neither blocks nor log segments found. Creating new log.
I20260812 16:33:03.599167 31076 log.cc:824] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab: Log is configured to *not* fsync() on all Append() calls
I20260812 16:33:03.599995 31076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab: No bootstrap required, opened a new log
I20260812 16:33:03.601594 31076 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb172205a1c4f6e9daaa50df34de8ab" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45297 } }
I20260812 16:33:03.601740 31076 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:33:03.601766 31076 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb172205a1c4f6e9daaa50df34de8ab, State: Initialized, Role: FOLLOWER
I20260812 16:33:03.601934 31076 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb172205a1c4f6e9daaa50df34de8ab" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45297 } }
I20260812 16:33:03.602010 31076 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:33:03.602034 31076 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:33:03.602084 31076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:33:03.602627 31076 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb172205a1c4f6e9daaa50df34de8ab" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45297 } }
I20260812 16:33:03.602738 31076 leader_election.cc:302] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fb172205a1c4f6e9daaa50df34de8ab; no voters:
I20260812 16:33:03.602941 31076 leader_election.cc:288] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:33:03.603049 31079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:33:03.603238 31076 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:33:03.603240 31079 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab [term 1 LEADER]: Becoming Leader. State: Replica: 7fb172205a1c4f6e9daaa50df34de8ab, State: Running, Role: LEADER
I20260812 16:33:03.603452 31079 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb172205a1c4f6e9daaa50df34de8ab" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45297 } }
I20260812 16:33:03.603948 31083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fb172205a1c4f6e9daaa50df34de8ab. Latest consensus state: current_term: 1 leader_uuid: "7fb172205a1c4f6e9daaa50df34de8ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb172205a1c4f6e9daaa50df34de8ab" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45297 } } }
I20260812 16:33:03.604048 31082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7fb172205a1c4f6e9daaa50df34de8ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb172205a1c4f6e9daaa50df34de8ab" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45297 } } }
I20260812 16:33:03.604347 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 30999
I20260812 16:33:03.604468 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:33:03.604467 31082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab [sys.catalog]: This master's current role is: LEADER
I20260812 16:33:03.604722 20409 external_mini_cluster.cc:1428] Setting key fc06d052075a516fa11dff2b999c020a
I20260812 16:33:03.604753 31094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:33:03.605067 31083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab [sys.catalog]: This master's current role is: LEADER
I20260812 16:33:03.605381 31094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:33:03.607488 31094 catalog_manager.cc:1347] Generated new cluster ID: 47123e24a1d64692b66af9a64cc1fd0c
I20260812 16:33:03.607556 31094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:33:03.610646 31100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:03.605247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34160 (local address 127.19.238.126:45297)
0812 16:33:03.605513 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:03.605522 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:03.605569 (+ 47us) server_negotiation.cc:407] Connection header received
0812 16:33:03.605687 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:03.605694 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:03.605771 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:03.605913 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:03.606143 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:03.607333 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:03.608437 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:03.609074 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:03.609209 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:03.609906 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:33:03.609910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:33:03.609910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:33:03.610130 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:33:03.610173 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:33:03.610187 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:33:03.610262 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:03.610331 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 16:33:03.610412 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":84,"threads_started":1}
I20260812 16:33:03.611876 31094 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:33:03.612320 31094 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:33:03.625998 31094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7fb172205a1c4f6e9daaa50df34de8ab: Generated new TSK 0
I20260812 16:33:03.626284 31094 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:33:03.632753 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:45297
--builtin_ntp_servers=127.19.238.84:46653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 16:33:03.718984 31102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:33:03.719202 31102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:33:03.719225 31102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:33:03.720474 31102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:33:03.720515 31102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:33:03.720541 31102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:33:03.721881 31102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:46653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:45297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:33:03.722137 31102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:33:03.722457 31102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:33:03.728617 31109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:03.728713 31111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:03.728626 31108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:33:03.728972 31102 server_base.cc:1034] running on GCE node
I20260812 16:33:03.729182 31102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:33:03.729426 31102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:33:03.730621 31102 hybrid_clock.cc:648] HybridClock initialized: now 1786552383730602 us; error 29 us; skew 500 ppm
I20260812 16:33:03.732043 31102 webserver.cc:460] Webserver started at http://127.19.238.65:40085/ using document root <none> and password file <none>
I20260812 16:33:03.732293 31102 fs_manager.cc:359] Metadata directory not provided
I20260812 16:33:03.732366 31102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:33:03.732525 31102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:33:03.733659 31102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "306758bcb42f45c1877c7e1a9ca5d260"
format_stamp: "Formatted at 2026-08-12 16:33:03 on dist-test-slave-mvvj"
server_key: "9afe40c7e13e03eb1fdd5f7c8d59a02d"
server_key_iv: "5b21fc50018f4f4b553c3c9dff745951"
server_key_version: "encryptionkey@0"
I20260812 16:33:03.734048 31102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "306758bcb42f45c1877c7e1a9ca5d260"
format_stamp: "Formatted at 2026-08-12 16:33:03 on dist-test-slave-mvvj"
server_key: "9afe40c7e13e03eb1fdd5f7c8d59a02d"
server_key_iv: "5b21fc50018f4f4b553c3c9dff745951"
server_key_version: "encryptionkey@0"
I20260812 16:33:03.735477 31102 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 16:33:03.736330 31117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:03.736496 31102 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:03.736604 31102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "306758bcb42f45c1877c7e1a9ca5d260"
format_stamp: "Formatted at 2026-08-12 16:33:03 on dist-test-slave-mvvj"
server_key: "9afe40c7e13e03eb1fdd5f7c8d59a02d"
server_key_iv: "5b21fc50018f4f4b553c3c9dff745951"
server_key_version: "encryptionkey@0"
I20260812 16:33:03.736681 31102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:33:03.759781 31102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:33:03.760210 31102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:33:03.760380 31102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:33:03.760635 31102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:33:03.761018 31102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:33:03.761078 31102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:03.761122 31102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:33:03.761171 31102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:03.767439 31102 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:33995
I20260812 16:33:03.767496 31243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:33995 every 8 connection(s)
I20260812 16:33:03.767951 31102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:33:03.769060 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 31102
I20260812 16:33:03.769161 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:33:03.769399 20409 external_mini_cluster.cc:1428] Setting key b0d46aedcb1429c135f77556a7738a07
I20260812 16:33:03.770568 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:45297
--builtin_ntp_servers=127.19.238.84:46653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 16:33:03.774374 31100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:03.768777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:58553 (local address 127.19.238.126:45297)
0812 16:33:03.768869 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:03.768871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:03.769154 (+ 283us) server_negotiation.cc:407] Connection header received
0812 16:33:03.769898 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:03.769899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:03.769912 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:03.769954 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:03.770861 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:03.771423 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:03.772829 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:03.773038 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:03.773136 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:03.774065 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:33:03.774067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:33:03.774067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:33:03.774072 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:33:03.774088 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:33:03.774093 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:33:03.774166 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:03.774254 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 16:33:03.774291 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 16:33:03.774614 31246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:03.768728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:45297 (local address 127.19.238.65:58553)
0812 16:33:03.768977 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:03.769041 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:03.769764 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:03.770033 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:03.770037 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:03.770048 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:03.770763 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:03.770771 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:03.771528 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:03.771529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:03.772726 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:03.772729 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:03.773150 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:03.773151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:03.773300 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:03.773897 (+ 597us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:33:03.773898 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:03.773928 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:33:03.773953 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:33:03.774168 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:33:03.774177 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:33:03.774180 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:33:03.774257 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 16:33:03.774364 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":126,"threads_started":1}
I20260812 16:33:03.775115 31244 heartbeater.cc:347] Connected to a master server at 127.19.238.126:45297
I20260812 16:33:03.775235 31244 heartbeater.cc:464] Registering TS with master...
I20260812 16:33:03.775465 31244 heartbeater.cc:511] Master 127.19.238.126:45297 requested a full tablet report, sending...
I20260812 16:33:03.776059 31036 ts_manager.cc:194] Registered new tserver with Master: 306758bcb42f45c1877c7e1a9ca5d260 (127.19.238.65:33995)
I20260812 16:33:03.776691 31036 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.238.65:58553
W20260812 16:33:03.856459 31247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:33:03.856673 31247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:33:03.856694 31247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:33:03.857947 31247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:33:03.857993 31247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:33:03.858019 31247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:33:03.859382 31247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:46653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:45297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:33:03.859648 31247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:33:03.860039 31247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:33:03.865288 31256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:03.865252 31252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:03.865408 31253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:33:03.865473 31247 server_base.cc:1034] running on GCE node
I20260812 16:33:03.865787 31247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:33:03.866094 31247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:33:03.867257 31247 hybrid_clock.cc:648] HybridClock initialized: now 1786552383867235 us; error 34 us; skew 500 ppm
I20260812 16:33:03.868738 31247 webserver.cc:460] Webserver started at http://127.19.238.66:44409/ using document root <none> and password file <none>
I20260812 16:33:03.869031 31247 fs_manager.cc:359] Metadata directory not provided
I20260812 16:33:03.869110 31247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:33:03.869272 31247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:33:03.870424 31247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "907288df1bca4d91988107501487fd23"
format_stamp: "Formatted at 2026-08-12 16:33:03 on dist-test-slave-mvvj"
server_key: "42fae2a97594495558b13609ee284fc2"
server_key_iv: "5d81a3bab69998825092cf7e4c609e51"
server_key_version: "encryptionkey@0"
I20260812 16:33:03.870807 31247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "907288df1bca4d91988107501487fd23"
format_stamp: "Formatted at 2026-08-12 16:33:03 on dist-test-slave-mvvj"
server_key: "42fae2a97594495558b13609ee284fc2"
server_key_iv: "5d81a3bab69998825092cf7e4c609e51"
server_key_version: "encryptionkey@0"
I20260812 16:33:03.872272 31247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:33:03.873064 31262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:03.873240 31247 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:33:03.873347 31247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "907288df1bca4d91988107501487fd23"
format_stamp: "Formatted at 2026-08-12 16:33:03 on dist-test-slave-mvvj"
server_key: "42fae2a97594495558b13609ee284fc2"
server_key_iv: "5d81a3bab69998825092cf7e4c609e51"
server_key_version: "encryptionkey@0"
I20260812 16:33:03.873431 31247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:33:03.896466 31247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:33:03.896819 31247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:33:03.896984 31247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:33:03.897244 31247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:33:03.897607 31247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:33:03.897663 31247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:03.897709 31247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:33:03.897735 31247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:03.903985 31247 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:44915
I20260812 16:33:03.904033 31391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:44915 every 8 connection(s)
I20260812 16:33:03.904469 31247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:33:03.906491 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 31247
I20260812 16:33:03.906631 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:33:03.906821 20409 external_mini_cluster.cc:1428] Setting key 68d0c8835fbe637f729b1c23c40265e8
I20260812 16:33:03.907960 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:45297
--builtin_ntp_servers=127.19.238.84:46653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 16:33:03.911274 31100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:03.905259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:39421 (local address 127.19.238.126:45297)
0812 16:33:03.905430 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:03.905432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:03.905651 (+ 219us) server_negotiation.cc:407] Connection header received
0812 16:33:03.906287 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:03.906290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:03.906304 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:03.906375 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:03.907244 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:03.907906 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:03.909513 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:03.909850 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:03.909936 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:03.910984 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:33:03.910986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:33:03.910986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:33:03.910991 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:33:03.911008 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:33:03.911012 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:33:03.911068 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:03.911183 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 16:33:03.911212 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 16:33:03.911603 31394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:03.905243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:45297 (local address 127.19.238.66:39421)
0812 16:33:03.905501 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:03.905548 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:03.906157 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:03.906453 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:03.906457 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:03.906467 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:03.907121 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:03.907128 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:03.908010 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:03.908012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:03.909409 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:03.909413 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:03.909967 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:03.909969 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:03.910106 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:03.910771 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:33:03.910771 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:03.910828 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:33:03.910862 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:33:03.911091 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:33:03.911098 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:33:03.911101 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:33:03.911176 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 16:33:03.911302 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":97,"threads_started":1}
I20260812 16:33:03.912213 31392 heartbeater.cc:347] Connected to a master server at 127.19.238.126:45297
I20260812 16:33:03.912343 31392 heartbeater.cc:464] Registering TS with master...
I20260812 16:33:03.912576 31392 heartbeater.cc:511] Master 127.19.238.126:45297 requested a full tablet report, sending...
I20260812 16:33:03.913036 31036 ts_manager.cc:194] Registered new tserver with Master: 907288df1bca4d91988107501487fd23 (127.19.238.66:44915)
I20260812 16:33:03.913420 31036 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.238.66:39421
W20260812 16:33:03.994417 31395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:33:03.994639 31395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:33:03.994664 31395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:33:03.995886 31395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:33:03.995931 31395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:33:03.995956 31395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:33:03.997347 31395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:46653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:45297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:33:03.997619 31395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:33:03.997942 31395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:33:04.003276 31404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:04.003242 31401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:04.003263 31402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:33:04.003729 31395 server_base.cc:1034] running on GCE node
I20260812 16:33:04.004082 31395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:33:04.004382 31395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:33:04.005564 31395 hybrid_clock.cc:648] HybridClock initialized: now 1786552384005552 us; error 33 us; skew 500 ppm
I20260812 16:33:04.007038 31395 webserver.cc:460] Webserver started at http://127.19.238.67:33013/ using document root <none> and password file <none>
I20260812 16:33:04.007305 31395 fs_manager.cc:359] Metadata directory not provided
I20260812 16:33:04.007383 31395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:33:04.007541 31395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:33:04.008774 31395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "d556c1f4d6224999aa005a540648f352"
format_stamp: "Formatted at 2026-08-12 16:33:04 on dist-test-slave-mvvj"
server_key: "9041ea9017578beb7f06f13f5501847f"
server_key_iv: "71b0d5cde7359b0fc0b283cec489ab95"
server_key_version: "encryptionkey@0"
I20260812 16:33:04.009171 31395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "d556c1f4d6224999aa005a540648f352"
format_stamp: "Formatted at 2026-08-12 16:33:04 on dist-test-slave-mvvj"
server_key: "9041ea9017578beb7f06f13f5501847f"
server_key_iv: "71b0d5cde7359b0fc0b283cec489ab95"
server_key_version: "encryptionkey@0"
I20260812 16:33:04.010625 31395 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:33:04.011456 31410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:04.011651 31395 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:33:04.011759 31395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "d556c1f4d6224999aa005a540648f352"
format_stamp: "Formatted at 2026-08-12 16:33:04 on dist-test-slave-mvvj"
server_key: "9041ea9017578beb7f06f13f5501847f"
server_key_iv: "71b0d5cde7359b0fc0b283cec489ab95"
server_key_version: "encryptionkey@0"
I20260812 16:33:04.011870 31395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:33:04.027976 31395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:33:04.028363 31395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:33:04.028534 31395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:33:04.028784 31395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:33:04.029150 31395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:33:04.029206 31395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:04.029241 31395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:33:04.029270 31395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:04.035362 31395 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:40839
I20260812 16:33:04.035413 31540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:40839 every 8 connection(s)
I20260812 16:33:04.035759 31395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:33:04.041528 31100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:04.036642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:37819 (local address 127.19.238.126:45297)
0812 16:33:04.036710 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:04.036712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:04.036974 (+ 262us) server_negotiation.cc:407] Connection header received
0812 16:33:04.037550 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:04.037551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:04.037564 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:04.037646 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:04.038495 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:04.039124 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:04.040196 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:04.040387 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:04.040446 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:04.041277 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:33:04.041279 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:33:04.041279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:33:04.041284 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:33:04.041298 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:33:04.041302 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:33:04.041348 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:04.041436 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 16:33:04.041462 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 16:33:04.041734 31544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:04.036645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:45297 (local address 127.19.238.67:37819)
0812 16:33:04.036842 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:04.036885 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:04.037391 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:04.037796 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:04.037800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:04.037809 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:04.038357 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:04.038363 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:04.039217 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:04.039218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:04.040113 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:04.040116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:04.040463 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:04.040464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:04.040574 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:04.041136 (+ 562us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:33:04.041136 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:04.041167 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:33:04.041191 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:33:04.041363 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:33:04.041368 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:33:04.041370 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:33:04.041423 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 16:33:04.041511 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
I20260812 16:33:04.042320 31542 heartbeater.cc:347] Connected to a master server at 127.19.238.126:45297
I20260812 16:33:04.042407 31542 heartbeater.cc:464] Registering TS with master...
I20260812 16:33:04.042641 31542 heartbeater.cc:511] Master 127.19.238.126:45297 requested a full tablet report, sending...
I20260812 16:33:04.043073 31036 ts_manager.cc:194] Registered new tserver with Master: d556c1f4d6224999aa005a540648f352 (127.19.238.67:40839)
I20260812 16:33:04.043406 31036 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.238.67:37819
I20260812 16:33:04.044734 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 31395
I20260812 16:33:04.044811 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:33:04.044970 20409 external_mini_cluster.cc:1428] Setting key ba6bc0ba3d7da1c1552cdb157f2bae55
I20260812 16:33:04.045706 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 16:33:04.047605 31100 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:33:04.047023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34166 (local address 127.19.238.126:45297)
0812 16:33:04.047087 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:04.047089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:04.047213 (+ 124us) server_negotiation.cc:407] Connection header received
0812 16:33:04.047293 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:04.047293 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:04.047306 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:04.047375 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:04.047534 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34166: BlockingRecv error: recv got EOF from 127.0.0.1:34166 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:33:04.047715 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 31102
I20260812 16:33:04.060956 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 31247
I20260812 16:33:04.068019 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 31395
I20260812 16:33:04.075563 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 30999
2026-08-12T16:33:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (639 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T16:33:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:33:04Z Disabled control of system clock
I20260812 16:33:04.105871 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:43427
--webserver_interface=127.19.238.126
--webserver_port=0
--builtin_ntp_servers=127.19.238.84:46061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:43427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 16:33:04.188504 31553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:33:04.188730 31553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:33:04.188753 31553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:33:04.189844 31553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:33:04.189880 31553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:33:04.189893 31553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:33:04.189905 31553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:33:04.189918 31553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:33:04.191304 31553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:46061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:43427
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:43427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:33:04.191517 31553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:33:04.191758 31553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:33:04.196802 31561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:04.196771 31558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:04.196832 31559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:33:04.197093 31553 server_base.cc:1034] running on GCE node
I20260812 16:33:04.197356 31553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:33:04.197659 31553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:33:04.198836 31553 hybrid_clock.cc:648] HybridClock initialized: now 1786552384198876 us; error 98 us; skew 500 ppm
I20260812 16:33:04.200305 31553 webserver.cc:460] Webserver started at http://127.19.238.126:39495/ using document root <none> and password file <none>
I20260812 16:33:04.200558 31553 fs_manager.cc:359] Metadata directory not provided
I20260812 16:33:04.200630 31553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:33:04.200788 31553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:33:04.201954 31553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "01270e2a6a1042f38217b7d9d834a4c9"
format_stamp: "Formatted at 2026-08-12 16:33:04 on dist-test-slave-mvvj"
server_key: "fc68bbec05ec54bf338ceaf8fcc8f981"
server_key_iv: "af16b99bb9b11257600540d0b253cc9a"
server_key_version: "encryptionkey@0"
I20260812 16:33:04.202337 31553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "01270e2a6a1042f38217b7d9d834a4c9"
format_stamp: "Formatted at 2026-08-12 16:33:04 on dist-test-slave-mvvj"
server_key: "fc68bbec05ec54bf338ceaf8fcc8f981"
server_key_iv: "af16b99bb9b11257600540d0b253cc9a"
server_key_version: "encryptionkey@0"
I20260812 16:33:04.203723 31553 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:33:04.204649 31568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:04.204864 31553 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:33:04.204982 31553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "01270e2a6a1042f38217b7d9d834a4c9"
format_stamp: "Formatted at 2026-08-12 16:33:04 on dist-test-slave-mvvj"
server_key: "fc68bbec05ec54bf338ceaf8fcc8f981"
server_key_iv: "af16b99bb9b11257600540d0b253cc9a"
server_key_version: "encryptionkey@0"
I20260812 16:33:04.205072 31553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:33:04.208748 31553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:33:04.209060 31553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:33:04.209210 31553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:33:04.213584 31553 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:43427
I20260812 16:33:04.213641 31625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:43427 every 8 connection(s)
I20260812 16:33:04.214056 31553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:33:04.214751 31626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:33:04.217732 31626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9: Bootstrap starting.
I20260812 16:33:04.218293 31626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9: Neither blocks nor log segments found. Creating new log.
I20260812 16:33:04.218568 31626 log.cc:824] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9: Log is configured to *not* fsync() on all Append() calls
I20260812 16:33:04.219281 31626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9: No bootstrap required, opened a new log
I20260812 16:33:04.220611 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 31553
I20260812 16:33:04.220734 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:33:04.220968 20409 external_mini_cluster.cc:1428] Setting key d64291c62fc67e9519a6c0d2d6e2d3ab
I20260812 16:33:04.221798 31626 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01270e2a6a1042f38217b7d9d834a4c9" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 43427 } }
I20260812 16:33:04.222003 31626 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:33:04.222049 31626 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01270e2a6a1042f38217b7d9d834a4c9, State: Initialized, Role: FOLLOWER
I20260812 16:33:04.222191 31626 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01270e2a6a1042f38217b7d9d834a4c9" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 43427 } }
I20260812 16:33:04.222275 31626 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:33:04.222313 31626 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:33:04.222361 31626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:33:04.223179 31626 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01270e2a6a1042f38217b7d9d834a4c9" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 43427 } }
I20260812 16:33:04.223297 31626 leader_election.cc:302] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01270e2a6a1042f38217b7d9d834a4c9; no voters:
I20260812 16:33:04.223479 31626 leader_election.cc:288] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:33:04.223506 31631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:33:04.223673 31631 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9 [term 1 LEADER]: Becoming Leader. State: Replica: 01270e2a6a1042f38217b7d9d834a4c9, State: Running, Role: LEADER
I20260812 16:33:04.223762 31626 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:33:04.223789 31631 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01270e2a6a1042f38217b7d9d834a4c9" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 43427 } }
I20260812 16:33:04.224175 31631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01270e2a6a1042f38217b7d9d834a4c9. Latest consensus state: current_term: 1 leader_uuid: "01270e2a6a1042f38217b7d9d834a4c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01270e2a6a1042f38217b7d9d834a4c9" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 43427 } } }
I20260812 16:33:04.224279 31631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9 [sys.catalog]: This master's current role is: LEADER
I20260812 16:33:04.224311 31630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:04.221660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43790 (local address 127.19.238.126:43427)
0812 16:33:04.222110 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:04.222119 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:04.222188 (+ 69us) server_negotiation.cc:407] Connection header received
0812 16:33:04.222312 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:04.222320 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:04.222379 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:04.222532 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:04.223317 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:33:04.223320 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:33:04.223321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:33:04.223325 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:33:04.223369 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:33:04.223384 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:33:04.223452 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:04.223536 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 16:33:04.223692 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":114,"threads_started":1}
I20260812 16:33:04.224186 31633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01270e2a6a1042f38217b7d9d834a4c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01270e2a6a1042f38217b7d9d834a4c9" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 43427 } } }
I20260812 16:33:04.224814 31633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9 [sys.catalog]: This master's current role is: LEADER
W20260812 16:33:04.225678 31649 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:33:04.225725 31649 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:33:04.225765 31639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:33:04.225943 31639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:33:04.227401 31639 catalog_manager.cc:1347] Generated new cluster ID: 27ed72241ad748529272aeea981b6b7f
I20260812 16:33:04.227458 31639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:33:04.233173 31639 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:33:04.233347 31639 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:33:04.239274 31639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01270e2a6a1042f38217b7d9d834a4c9: Generated new TSK 0
I20260812 16:33:04.239542 31639 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:33:04.244896 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:43427
--builtin_ntp_servers=127.19.238.84:46061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 16:33:04.329594 31653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:33:04.329811 31653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:33:04.329834 31653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:33:04.331081 31653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:33:04.331123 31653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:33:04.331151 31653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:33:04.332572 31653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:46061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:43427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:33:04.332828 31653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:33:04.333143 31653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:33:04.338208 31658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:04.338362 31659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:04.338258 31662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:33:04.338639 31653 server_base.cc:1034] running on GCE node
I20260812 16:33:04.338953 31653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:33:04.339325 31653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:33:04.340519 31653 hybrid_clock.cc:648] HybridClock initialized: now 1786552384340496 us; error 36 us; skew 500 ppm
I20260812 16:33:04.341936 31653 webserver.cc:460] Webserver started at http://127.19.238.65:32901/ using document root <none> and password file <none>
I20260812 16:33:04.342211 31653 fs_manager.cc:359] Metadata directory not provided
I20260812 16:33:04.342257 31653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:33:04.342387 31653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:33:04.343549 31653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "1f878c4ec1f742818c7f122535af8308"
format_stamp: "Formatted at 2026-08-12 16:33:04 on dist-test-slave-mvvj"
server_key: "0c43e943666e1d0b0e5df0f06d5e8309"
server_key_iv: "9debab85868280838ffbe1dbcb94f8d1"
server_key_version: "encryptionkey@0"
I20260812 16:33:04.343998 31653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "1f878c4ec1f742818c7f122535af8308"
format_stamp: "Formatted at 2026-08-12 16:33:04 on dist-test-slave-mvvj"
server_key: "0c43e943666e1d0b0e5df0f06d5e8309"
server_key_iv: "9debab85868280838ffbe1dbcb94f8d1"
server_key_version: "encryptionkey@0"
I20260812 16:33:04.345368 31653 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:33:04.346700 31668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:04.346882 31653 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:33:04.347002 31653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "1f878c4ec1f742818c7f122535af8308"
format_stamp: "Formatted at 2026-08-12 16:33:04 on dist-test-slave-mvvj"
server_key: "0c43e943666e1d0b0e5df0f06d5e8309"
server_key_iv: "9debab85868280838ffbe1dbcb94f8d1"
server_key_version: "encryptionkey@0"
I20260812 16:33:04.347085 31653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:33:04.350683 31653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:33:04.350963 31653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:33:04.351105 31653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:33:04.351338 31653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:33:04.351671 31653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:33:04.351733 31653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:04.351775 31653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:33:04.351802 31653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:04.357966 31653 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:45909
I20260812 16:33:04.358016 31795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:45909 every 8 connection(s)
I20260812 16:33:04.358376 31653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:33:04.360687 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 31653
I20260812 16:33:04.360781 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:33:04.361028 20409 external_mini_cluster.cc:1428] Setting key 2669c3694c4437212477dada4774a923
I20260812 16:33:04.361828 31630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:04.359125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:35679 (local address 127.19.238.126:43427)
0812 16:33:04.359193 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:04.359195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:04.359466 (+ 271us) server_negotiation.cc:407] Connection header received
0812 16:33:04.360069 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:04.360071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:04.360078 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:04.360136 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:04.361521 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:33:04.361522 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:33:04.361523 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:33:04.361527 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:33:04.361544 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:33:04.361549 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:33:04.361616 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:04.361725 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 16:33:04.361743 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 16:33:04.361987 31798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:04.359127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:43427 (local address 127.19.238.65:35679)
0812 16:33:04.359332 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:04.359370 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:04.359957 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:04.360213 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:04.360217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:04.360224 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:04.361360 (+ 1136us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:33:04.361360 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:04.361405 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:33:04.361436 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:33:04.361635 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:33:04.361639 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:33:04.361642 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:33:04.361720 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 16:33:04.361779 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":105,"threads_started":1}
I20260812 16:33:04.362265 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:43427
--builtin_ntp_servers=127.19.238.84:46061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 16:33:04.364140 31796 heartbeater.cc:347] Connected to a master server at 127.19.238.126:43427
I20260812 16:33:04.364248 31796 heartbeater.cc:464] Registering TS with master...
I20260812 16:33:04.364399 31796 heartbeater.cc:511] Master 127.19.238.126:43427 requested a full tablet report, sending...
I20260812 16:33:04.364948 31587 ts_manager.cc:194] Registered new tserver with Master: 1f878c4ec1f742818c7f122535af8308 (127.19.238.65:45909)
W20260812 16:33:04.445904 31799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:33:04.446157 31799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:33:04.446195 31799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:33:04.447353 31799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:33:04.447428 31799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:33:04.447474 31799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:33:04.448926 31799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:46061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:43427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:33:04.449205 31799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:33:04.449517 31799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:33:04.455092 31808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:04.455092 31806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:04.455119 31805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:33:04.455411 31799 server_base.cc:1034] running on GCE node
I20260812 16:33:04.455617 31799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:33:04.455878 31799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:33:04.457051 31799 hybrid_clock.cc:648] HybridClock initialized: now 1786552384457029 us; error 30 us; skew 500 ppm
I20260812 16:33:04.458567 31799 webserver.cc:460] Webserver started at http://127.19.238.66:39547/ using document root <none> and password file <none>
I20260812 16:33:04.458817 31799 fs_manager.cc:359] Metadata directory not provided
I20260812 16:33:04.458863 31799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:33:04.459015 31799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:33:04.460238 31799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "9c951217ea4b4ebd99d6386dd17ff17d"
format_stamp: "Formatted at 2026-08-12 16:33:04 on dist-test-slave-mvvj"
server_key: "1be88da8065c471bea13d0da87e07884"
server_key_iv: "c16d65cf64ca10d38cb42fcfb3d17e7d"
server_key_version: "encryptionkey@0"
I20260812 16:33:04.460616 31799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "9c951217ea4b4ebd99d6386dd17ff17d"
format_stamp: "Formatted at 2026-08-12 16:33:04 on dist-test-slave-mvvj"
server_key: "1be88da8065c471bea13d0da87e07884"
server_key_iv: "c16d65cf64ca10d38cb42fcfb3d17e7d"
server_key_version: "encryptionkey@0"
I20260812 16:33:04.461966 31799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:33:04.462725 31815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:04.462914 31799 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:33:04.463006 31799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "9c951217ea4b4ebd99d6386dd17ff17d"
format_stamp: "Formatted at 2026-08-12 16:33:04 on dist-test-slave-mvvj"
server_key: "1be88da8065c471bea13d0da87e07884"
server_key_iv: "c16d65cf64ca10d38cb42fcfb3d17e7d"
server_key_version: "encryptionkey@0"
I20260812 16:33:04.463100 31799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:33:04.466514 31799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:33:04.466789 31799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:33:04.466930 31799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:33:04.467144 31799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:33:04.467463 31799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:33:04.467518 31799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:04.467550 31799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:33:04.467577 31799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:04.473371 31799 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:33549
I20260812 16:33:04.473444 31947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:33549 every 8 connection(s)
I20260812 16:33:04.473793 31799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:33:04.476835 31630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:04.474558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:43451 (local address 127.19.238.126:43427)
0812 16:33:04.474630 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:04.474632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:04.474864 (+ 232us) server_negotiation.cc:407] Connection header received
0812 16:33:04.475359 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:04.475360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:04.475365 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:04.475429 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:04.476621 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:33:04.476622 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:33:04.476623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:33:04.476627 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:33:04.476643 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:33:04.476647 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:33:04.476705 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:04.476766 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 16:33:04.476779 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 16:33:04.477101 31951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:04.474532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:43427 (local address 127.19.238.66:43451)
0812 16:33:04.474744 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:04.474787 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:04.475270 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:04.475483 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:04.475487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:04.475495 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:04.476446 (+ 951us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:33:04.476447 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:04.476475 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:33:04.476503 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:33:04.476704 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:33:04.476706 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:33:04.476709 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:33:04.476762 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 16:33:04.476809 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":113,"threads_started":1}
I20260812 16:33:04.477578 31948 heartbeater.cc:347] Connected to a master server at 127.19.238.126:43427
I20260812 16:33:04.477661 31948 heartbeater.cc:464] Registering TS with master...
I20260812 16:33:04.477837 31948 heartbeater.cc:511] Master 127.19.238.126:43427 requested a full tablet report, sending...
I20260812 16:33:04.478210 31587 ts_manager.cc:194] Registered new tserver with Master: 9c951217ea4b4ebd99d6386dd17ff17d (127.19.238.66:33549)
I20260812 16:33:04.478327 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 31799
I20260812 16:33:04.478447 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:33:04.478760 20409 external_mini_cluster.cc:1428] Setting key 31c2a7822c766d31c039faf0adca52ae
I20260812 16:33:04.479650 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:43427
--builtin_ntp_servers=127.19.238.84:46061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 16:33:04.561734 31952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:33:04.561941 31952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:33:04.561964 31952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:33:04.563148 31952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:33:04.563189 31952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:33:04.563241 31952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:33:04.564595 31952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:46061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:43427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:33:04.564834 31952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:33:04.565124 31952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:33:04.570351 31959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:04.570362 31958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:04.570366 31962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:33:04.570796 31952 server_base.cc:1034] running on GCE node
I20260812 16:33:04.571026 31952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:33:04.571273 31952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:33:04.572436 31952 hybrid_clock.cc:648] HybridClock initialized: now 1786552384572416 us; error 31 us; skew 500 ppm
I20260812 16:33:04.573796 31952 webserver.cc:460] Webserver started at http://127.19.238.67:35893/ using document root <none> and password file <none>
I20260812 16:33:04.574048 31952 fs_manager.cc:359] Metadata directory not provided
I20260812 16:33:04.574121 31952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:33:04.574280 31952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:33:04.575436 31952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "e72fdb2478f040d1960f80eef7d1487e"
format_stamp: "Formatted at 2026-08-12 16:33:04 on dist-test-slave-mvvj"
server_key: "4cda165ef65bfa44419022377ff0280f"
server_key_iv: "99e8591f182ac1e2968ac250d1a8b60c"
server_key_version: "encryptionkey@0"
I20260812 16:33:04.575822 31952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "e72fdb2478f040d1960f80eef7d1487e"
format_stamp: "Formatted at 2026-08-12 16:33:04 on dist-test-slave-mvvj"
server_key: "4cda165ef65bfa44419022377ff0280f"
server_key_iv: "99e8591f182ac1e2968ac250d1a8b60c"
server_key_version: "encryptionkey@0"
I20260812 16:33:04.577250 31952 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:33:04.578086 31968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:04.578279 31952 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:33:04.578454 31952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "e72fdb2478f040d1960f80eef7d1487e"
format_stamp: "Formatted at 2026-08-12 16:33:04 on dist-test-slave-mvvj"
server_key: "4cda165ef65bfa44419022377ff0280f"
server_key_iv: "99e8591f182ac1e2968ac250d1a8b60c"
server_key_version: "encryptionkey@0"
I20260812 16:33:04.578552 31952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:33:04.582011 31952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:33:04.582278 31952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:33:04.582428 31952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:33:04.582645 31952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:33:04.582966 31952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:33:04.583024 31952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:04.583063 31952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:33:04.583092 31952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:04.588971 31952 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:34273
I20260812 16:33:04.589011 32098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:34273 every 8 connection(s)
I20260812 16:33:04.589373 31952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:33:04.592607 31630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:04.590162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:58289 (local address 127.19.238.126:43427)
0812 16:33:04.590241 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:04.590245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:04.590460 (+ 215us) server_negotiation.cc:407] Connection header received
0812 16:33:04.590941 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:04.590942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:04.590947 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:04.591003 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:04.592305 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:33:04.592307 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:33:04.592307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:33:04.592311 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:33:04.592329 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:33:04.592334 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:33:04.592407 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:04.592531 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 16:33:04.592545 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 16:33:04.592834 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:04.590115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:43427 (local address 127.19.238.67:58289)
0812 16:33:04.590323 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:04.590369 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:04.590854 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:04.591057 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:04.591060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:04.591068 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:04.592132 (+ 1064us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:33:04.592132 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:04.592162 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:33:04.592192 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:33:04.592435 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:33:04.592441 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:33:04.592444 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:33:04.592530 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 16:33:04.592583 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":100,"threads_started":1}
I20260812 16:33:04.593430 32099 heartbeater.cc:347] Connected to a master server at 127.19.238.126:43427
I20260812 16:33:04.593513 32099 heartbeater.cc:464] Registering TS with master...
I20260812 16:33:04.593624 32099 heartbeater.cc:511] Master 127.19.238.126:43427 requested a full tablet report, sending...
I20260812 16:33:04.594023 31587 ts_manager.cc:194] Registered new tserver with Master: e72fdb2478f040d1960f80eef7d1487e (127.19.238.67:34273)
I20260812 16:33:04.595072 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 31952
I20260812 16:33:04.595185 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:33:04.595391 20409 external_mini_cluster.cc:1428] Setting key 66f03c74dc71d06e6bba081d55da0225
I20260812 16:33:04.596413 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 16:33:04.598218 31630 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:33:04.597769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43798 (local address 127.19.238.126:43427)
0812 16:33:04.597832 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:04.597833 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:04.597945 (+ 112us) server_negotiation.cc:407] Connection header received
0812 16:33:04.598012 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:04.598013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:04.598017 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:04.598066 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:04.598163 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43798: BlockingRecv error: recv got EOF from 127.0.0.1:43798 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:33:04.598284 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 31653
I20260812 16:33:04.608120 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 31799
I20260812 16:33:04.620795 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 31952
I20260812 16:33:04.628705 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 31553
2026-08-12T16:33:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (552 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:33:04 dist-test-slave-mvvj krb5kdc[32110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:33:04 dist-test-slave-mvvj krb5kdc[32110](info): set up 2 sockets
Aug 12 16:33:04 dist-test-slave-mvvj krb5kdc[32110](info): commencing operation
krb5kdc: starting...
W20260812 16:33:05.745884 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:33:05 dist-test-slave-mvvj krb5kdc[32110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:33:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:33:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:33:05.882542 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:42617
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:42617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:33:05.968375 32125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:33:05.968605 32125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:33:05.968631 32125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:33:05.969798 32125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:33:05.969844 32125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:33:05.969858 32125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:33:05.969871 32125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:33:05.969885 32125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:33:05.971365 32125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:42617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:42617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:33:05.971593 32125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:33:05.971781 32125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:33:05.976938 32130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:05.976925 32134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:05.976925 32131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:33:05.977269 32125 server_base.cc:1034] running on GCE node
I20260812 16:33:05.977541 32125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:33:05.977864 32125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:33:05.979041 32125 hybrid_clock.cc:648] HybridClock initialized: now 1786552385979026 us; error 50 us; skew 500 ppm
Aug 12 16:33:05 dist-test-slave-mvvj krb5kdc[32110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552385, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:33:05.981715 32125 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:33:05.982254 32125 webserver.cc:460] Webserver started at http://127.19.238.126:44059/ using document root <none> and password file <none>
I20260812 16:33:05.982496 32125 fs_manager.cc:359] Metadata directory not provided
I20260812 16:33:05.982574 32125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:33:05.982728 32125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:33:05.983923 32125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "c9de6bb506ef4408bd53a030d79f0010"
format_stamp: "Formatted at 2026-08-12 16:33:05 on dist-test-slave-mvvj"
server_key: "d340037f43937faeddafd834e9f7095a"
server_key_iv: "6027f2235abf3e8110205e608fdd322c"
server_key_version: "encryptionkey@0"
I20260812 16:33:05.984323 32125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "c9de6bb506ef4408bd53a030d79f0010"
format_stamp: "Formatted at 2026-08-12 16:33:05 on dist-test-slave-mvvj"
server_key: "d340037f43937faeddafd834e9f7095a"
server_key_iv: "6027f2235abf3e8110205e608fdd322c"
server_key_version: "encryptionkey@0"
I20260812 16:33:05.985780 32125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:33:05.986662 32141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:05.986850 32125 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:33:05.986964 32125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "c9de6bb506ef4408bd53a030d79f0010"
format_stamp: "Formatted at 2026-08-12 16:33:05 on dist-test-slave-mvvj"
server_key: "d340037f43937faeddafd834e9f7095a"
server_key_iv: "6027f2235abf3e8110205e608fdd322c"
server_key_version: "encryptionkey@0"
I20260812 16:33:05.987046 32125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:33:05.995731 32125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:33:05.997957 32125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:33:05.998147 32125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:33:06.002574 32125 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:42617
I20260812 16:33:06.002614 32202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:42617 every 8 connection(s)
I20260812 16:33:06.003001 32125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:33:06.004110 32203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:33:06.006541 32203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010: Bootstrap starting.
I20260812 16:33:06.007140 32203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010: Neither blocks nor log segments found. Creating new log.
I20260812 16:33:06.007395 32203 log.cc:824] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010: Log is configured to *not* fsync() on all Append() calls
I20260812 16:33:06.008210 32203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010: No bootstrap required, opened a new log
I20260812 16:33:06.008677 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 32125
I20260812 16:33:06.008773 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:33:06.008997 20409 external_mini_cluster.cc:1428] Setting key f96a295569b95584f785f21ec3dd2370
I20260812 16:33:06.010815 32203 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9de6bb506ef4408bd53a030d79f0010" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42617 } }
I20260812 16:33:06.011057 32203 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:33:06.011102 32203 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9de6bb506ef4408bd53a030d79f0010, State: Initialized, Role: FOLLOWER
I20260812 16:33:06.011240 32203 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9de6bb506ef4408bd53a030d79f0010" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42617 } }
I20260812 16:33:06.011340 32203 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:33:06.011410 32203 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:33:06.011466 32203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:33:06.012245 32203 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9de6bb506ef4408bd53a030d79f0010" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42617 } }
I20260812 16:33:06.012344 32203 leader_election.cc:302] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9de6bb506ef4408bd53a030d79f0010; no voters:
I20260812 16:33:06.012565 32203 leader_election.cc:288] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:33:06.012611 32208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:33:06.012784 32208 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010 [term 1 LEADER]: Becoming Leader. State: Replica: c9de6bb506ef4408bd53a030d79f0010, State: Running, Role: LEADER
I20260812 16:33:06.012897 32203 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:33:06.012874 32208 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9de6bb506ef4408bd53a030d79f0010" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42617 } }
I20260812 16:33:06.013334 32208 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9de6bb506ef4408bd53a030d79f0010. Latest consensus state: current_term: 1 leader_uuid: "c9de6bb506ef4408bd53a030d79f0010" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9de6bb506ef4408bd53a030d79f0010" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42617 } } }
I20260812 16:33:06.013440 32208 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010 [sys.catalog]: This master's current role is: LEADER
I20260812 16:33:06.013340 32210 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9de6bb506ef4408bd53a030d79f0010" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9de6bb506ef4408bd53a030d79f0010" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42617 } } }
I20260812 16:33:06.013657 32210 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010 [sys.catalog]: This master's current role is: LEADER
I20260812 16:33:06.013665 32215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:33:06.014343 32215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:33:06.016708 32215 catalog_manager.cc:1347] Generated new cluster ID: 4a467515738e4374ac896a370dacbeb1
I20260812 16:33:06.016777 32215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:33:06 dist-test-slave-mvvj krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:33:06.021838 32207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.009589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35062 (local address 127.19.238.126:42617)
0812 16:33:06.009796 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:06.009805 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:06.009831 (+ 26us) server_negotiation.cc:407] Connection header received
0812 16:33:06.010324 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:06.010331 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:06.010381 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:06.010501 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:06.011360 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.012884 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.014046 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.014675 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.014782 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:06.018141 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:33:06.018146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:33:06.018146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:33:06.018160 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:33:06.020189 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:06.020599 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:06.020602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:06.020603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:06.020698 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:06.020935 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:06.020936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:06.020937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:06.021193 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:33:06.021396 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:06.021468 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 16:33:06.021615 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":99,"threads_started":1}
I20260812 16:33:06.037497 32215 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:33:06.038092 32215 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:33:06.043995 32215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010: Generated new TSK 0
I20260812 16:33:06.044243 32215 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
I20260812 16:33:06.079996 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42617
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:33:06.195402 32230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:33:06.195641 32230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:33:06.195669 32230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:33:06.196856 32230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:33:06.196895 32230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:33:06.196923 32230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:33:06.198400 32230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:33:06.198617 32230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:33:06.198823 32230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:33:06.204386 32236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:06.204417 32235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:06.204407 32239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:33:06.204416 32230 server_base.cc:1034] running on GCE node
I20260812 16:33:06.204841 32230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:33:06.205153 32230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:33:06.206362 32230 hybrid_clock.cc:648] HybridClock initialized: now 1786552386206340 us; error 38 us; skew 500 ppm
Aug 12 16:33:06 dist-test-slave-mvvj krb5kdc[32110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552386, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:33:06.209232 32230 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:33:06.209709 32230 webserver.cc:460] Webserver started at http://127.19.238.65:36589/ using document root <none> and password file <none>
I20260812 16:33:06.209971 32230 fs_manager.cc:359] Metadata directory not provided
I20260812 16:33:06.210024 32230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:33:06.210160 32230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:33:06.211275 32230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "a53223bf77c14ea0a1deb4ab79b13174"
format_stamp: "Formatted at 2026-08-12 16:33:06 on dist-test-slave-mvvj"
server_key: "361d9d6b3864f824bb3bb3143d77cde9"
server_key_iv: "48de9b6ee761a344ce521c753bfa9635"
server_key_version: "encryptionkey@0"
I20260812 16:33:06.211668 32230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "a53223bf77c14ea0a1deb4ab79b13174"
format_stamp: "Formatted at 2026-08-12 16:33:06 on dist-test-slave-mvvj"
server_key: "361d9d6b3864f824bb3bb3143d77cde9"
server_key_iv: "48de9b6ee761a344ce521c753bfa9635"
server_key_version: "encryptionkey@0"
I20260812 16:33:06.213296 32230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:33:06.214159 32247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:06.214342 32230 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:33:06.214429 32230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "a53223bf77c14ea0a1deb4ab79b13174"
format_stamp: "Formatted at 2026-08-12 16:33:06 on dist-test-slave-mvvj"
server_key: "361d9d6b3864f824bb3bb3143d77cde9"
server_key_iv: "48de9b6ee761a344ce521c753bfa9635"
server_key_version: "encryptionkey@0"
I20260812 16:33:06.214530 32230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:33:06.222443 32230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:33:06.224817 32230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:33:06.224937 32230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:33:06.225155 32230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:33:06.225479 32230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:33:06.225518 32230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:06.225543 32230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:33:06.225587 32230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:06.232767 32230 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:35813
I20260812 16:33:06.232808 32372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:35813 every 8 connection(s)
I20260812 16:33:06.233187 32230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:33:06.237257 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 32230
I20260812 16:33:06.237370 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:33:06.237577 20409 external_mini_cluster.cc:1428] Setting key 1c37b741124ed20e9111993e175de7c3
Aug 12 16:33:06 dist-test-slave-mvvj krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552386, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:33:06.244320 32207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.234027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:33789 (local address 127.19.238.126:42617)
0812 16:33:06.234114 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:06.234117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:06.234290 (+ 173us) server_negotiation.cc:407] Connection header received
0812 16:33:06.234798 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:06.234799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:06.234810 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:06.234866 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:06.235945 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.236457 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.237760 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.238071 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.238137 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:06.240750 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:33:06.240752 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:33:06.240753 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:33:06.240762 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:33:06.242338 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:06.243152 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:06.243154 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:06.243155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:06.243230 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:06.243565 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:06.243566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:06.243566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:06.243683 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:33:06.243741 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:06.244164 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 16:33:06.244251 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 16:33:06.244689 32377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.233966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:42617 (local address 127.19.238.65:33789)
0812 16:33:06.234164 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:06.234204 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:06.234710 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:06.234923 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:06.234926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:06.235556 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:06.235769 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:06.235774 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.236546 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.236547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:06.237631 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:06.237634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.238153 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.238153 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:06.238259 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:06.238874 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:33:06.238885 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:06.240643 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:33:06.242446 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:33:06.242456 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:33:06.242458 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:33:06.242919 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:33:06.243322 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:33:06.243323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:33:06.243324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:33:06.243486 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:33:06.243761 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:33:06.243769 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:33:06.244007 (+ 238us) client_negotiation.cc:769] Sending connection context
0812 16:33:06.244150 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 16:33:06.244364 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":130,"threads_started":1}
I20260812 16:33:06.245317 32373 heartbeater.cc:347] Connected to a master server at 127.19.238.126:42617
I20260812 16:33:06.245414 32373 heartbeater.cc:464] Registering TS with master...
I20260812 16:33:06.245621 32373 heartbeater.cc:511] Master 127.19.238.126:42617 requested a full tablet report, sending...
I20260812 16:33:06.246201 32162 ts_manager.cc:194] Registered new tserver with Master: a53223bf77c14ea0a1deb4ab79b13174 (127.19.238.65:35813)
I20260812 16:33:06.246842 32162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:33789
WARNING: no policy specified for kudu/127.19.238.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
I20260812 16:33:06.267766 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42617
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:33:06.353796 32380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:33:06.354046 32380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:33:06.354087 32380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:33:06.355284 32380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:33:06.355355 32380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:33:06.355405 32380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:33:06.357017 32380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:33:06.357321 32380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:33:06.357576 32380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:33:06.363030 32386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:06.363039 32385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:06.363039 32388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:33:06.363523 32380 server_base.cc:1034] running on GCE node
I20260812 16:33:06.363751 32380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:33:06.364048 32380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:33:06.365221 32380 hybrid_clock.cc:648] HybridClock initialized: now 1786552386365204 us; error 27 us; skew 500 ppm
Aug 12 16:33:06 dist-test-slave-mvvj krb5kdc[32110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552386, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:33:06.367919 32380 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:33:06.368391 32380 webserver.cc:460] Webserver started at http://127.19.238.66:34711/ using document root <none> and password file <none>
I20260812 16:33:06.368630 32380 fs_manager.cc:359] Metadata directory not provided
I20260812 16:33:06.368701 32380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:33:06.368858 32380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:33:06.369956 32380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "fae677c969f14608a287c00990eb584f"
format_stamp: "Formatted at 2026-08-12 16:33:06 on dist-test-slave-mvvj"
server_key: "e2ffa6f134997f801483c471ae67ce73"
server_key_iv: "df03ba914907256710f8af4c5b3682cd"
server_key_version: "encryptionkey@0"
I20260812 16:33:06.370357 32380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "fae677c969f14608a287c00990eb584f"
format_stamp: "Formatted at 2026-08-12 16:33:06 on dist-test-slave-mvvj"
server_key: "e2ffa6f134997f801483c471ae67ce73"
server_key_iv: "df03ba914907256710f8af4c5b3682cd"
server_key_version: "encryptionkey@0"
I20260812 16:33:06.371778 32380 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:33:06.372607 32396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:06.372785 32380 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 16:33:06.372896 32380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "fae677c969f14608a287c00990eb584f"
format_stamp: "Formatted at 2026-08-12 16:33:06 on dist-test-slave-mvvj"
server_key: "e2ffa6f134997f801483c471ae67ce73"
server_key_iv: "df03ba914907256710f8af4c5b3682cd"
server_key_version: "encryptionkey@0"
I20260812 16:33:06.372977 32380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:33:06.393761 32380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:33:06.396287 32380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:33:06.396483 32380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:33:06.396761 32380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:33:06.397100 32380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:33:06.397156 32380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:06.397198 32380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:33:06.397228 32380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:06.403294 32380 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:34145
I20260812 16:33:06.403350 32521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:34145 every 8 connection(s)
I20260812 16:33:06.403693 32380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:33:06.403986 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 32380
I20260812 16:33:06.404099 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:33:06.404336 20409 external_mini_cluster.cc:1428] Setting key c8d58cdb1eb355aa3ea9ee5b844de459
Aug 12 16:33:06 dist-test-slave-mvvj krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552386, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.238.67@KRBTEST.COM; defaulting to no policy
I20260812 16:33:06.413895 32207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.404733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:37935 (local address 127.19.238.126:42617)
0812 16:33:06.404790 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:06.404793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:06.404941 (+ 148us) server_negotiation.cc:407] Connection header received
0812 16:33:06.405557 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:06.405558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:06.405570 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:06.405665 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:06.406665 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.407177 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.408283 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.408500 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.408589 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:06.410882 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:33:06.410884 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:33:06.410884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:33:06.410896 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:33:06.412327 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:06.412829 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:06.412831 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:06.412832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:06.412919 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:06.413188 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:06.413189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:06.413190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:06.413343 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:33:06.413425 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:06.413749 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 16:33:06.413826 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 16:33:06.414084 32524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.404596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:42617 (local address 127.19.238.66:37935)
0812 16:33:06.404801 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:06.404856 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:06.405431 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:06.405742 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:06.405747 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:06.406354 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:06.406561 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:06.406566 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.407280 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.407281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:06.408191 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:06.408194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.408603 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.408604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:06.408731 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:06.409330 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:33:06.409339 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:06.410764 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:33:06.412420 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:33:06.412428 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:33:06.412430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:33:06.412738 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:33:06.412994 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:33:06.412994 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:33:06.412995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:33:06.413109 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:33:06.413417 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:33:06.413424 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:33:06.413570 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 16:33:06.413691 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 16:33:06.413842 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":119,"threads_started":1}
I20260812 16:33:06.414726 32522 heartbeater.cc:347] Connected to a master server at 127.19.238.126:42617
I20260812 16:33:06.414824 32522 heartbeater.cc:464] Registering TS with master...
I20260812 16:33:06.414969 32522 heartbeater.cc:511] Master 127.19.238.126:42617 requested a full tablet report, sending...
I20260812 16:33:06.415349 32162 ts_manager.cc:194] Registered new tserver with Master: fae677c969f14608a287c00990eb584f (127.19.238.66:34145)
I20260812 16:33:06.415742 32162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:37935
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
I20260812 16:33:06.436167 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42617
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:33:06.522310 32527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:33:06.522557 32527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:33:06.522599 32527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:33:06.523774 32527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:33:06.523844 32527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:33:06.523923 32527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:33:06.525430 32527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:33:06.525702 32527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:33:06.525951 32527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:33:06.531248 32533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:06.531257 32534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:33:06.531293 32527 server_base.cc:1034] running on GCE node
W20260812 16:33:06.531257 32536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:33:06.531733 32527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:33:06.532025 32527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:33:06.533169 32527 hybrid_clock.cc:648] HybridClock initialized: now 1786552386533151 us; error 41 us; skew 500 ppm
Aug 12 16:33:06 dist-test-slave-mvvj krb5kdc[32110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552386, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:33:06.535951 32527 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:33:06.536475 32527 webserver.cc:460] Webserver started at http://127.19.238.67:41275/ using document root <none> and password file <none>
I20260812 16:33:06.536734 32527 fs_manager.cc:359] Metadata directory not provided
I20260812 16:33:06.536782 32527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:33:06.536954 32527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:33:06.538060 32527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "317551a81b9c407c8bfee6a3ddc97c18"
format_stamp: "Formatted at 2026-08-12 16:33:06 on dist-test-slave-mvvj"
server_key: "2107b6907ce03c8d22f9357563fdbb12"
server_key_iv: "76b8e285d6f3ef7f13ae76d5f1ae7b56"
server_key_version: "encryptionkey@0"
I20260812 16:33:06.538461 32527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "317551a81b9c407c8bfee6a3ddc97c18"
format_stamp: "Formatted at 2026-08-12 16:33:06 on dist-test-slave-mvvj"
server_key: "2107b6907ce03c8d22f9357563fdbb12"
server_key_iv: "76b8e285d6f3ef7f13ae76d5f1ae7b56"
server_key_version: "encryptionkey@0"
I20260812 16:33:06.539988 32527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:33:06.540818 32544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:06.541006 32527 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:33:06.541103 32527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "317551a81b9c407c8bfee6a3ddc97c18"
format_stamp: "Formatted at 2026-08-12 16:33:06 on dist-test-slave-mvvj"
server_key: "2107b6907ce03c8d22f9357563fdbb12"
server_key_iv: "76b8e285d6f3ef7f13ae76d5f1ae7b56"
server_key_version: "encryptionkey@0"
I20260812 16:33:06.541198 32527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:33:06.554134 32527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:33:06.556397 32527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:33:06.556600 32527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:33:06.556849 32527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:33:06.557173 32527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:33:06.557228 32527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:06.557273 32527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:33:06.557300 32527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:06.563378 32527 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:39505
I20260812 16:33:06.563505 32677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:39505 every 8 connection(s)
I20260812 16:33:06.563827 32527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
Aug 12 16:33:06 dist-test-slave-mvvj krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552386, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:33:06.572398 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 32527
I20260812 16:33:06.572489 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:33:06.572698 20409 external_mini_cluster.cc:1428] Setting key 0b2d9cba56ca16a708d31f5f49d79138
I20260812 16:33:06.573935 32207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.564664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:49967 (local address 127.19.238.126:42617)
0812 16:33:06.564726 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:06.564729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:06.564920 (+ 191us) server_negotiation.cc:407] Connection header received
0812 16:33:06.565475 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:06.565476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:06.565490 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:06.565542 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:06.566541 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.567065 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.568152 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.568380 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.568459 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:06.570802 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:33:06.570803 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:33:06.570804 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:33:06.570813 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:33:06.572177 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:06.572687 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:06.572688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:06.572689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:06.572754 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:06.573094 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:06.573096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:06.573096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:06.573293 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:33:06.573439 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:06.573777 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 16:33:06.573870 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 16:33:06.574189 32680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.564598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:42617 (local address 127.19.238.67:49967)
0812 16:33:06.564769 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:06.564813 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:06.565360 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:06.565630 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:06.565634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:06.566234 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:06.566434 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:06.566439 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.567178 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.567179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:06.568062 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:06.568065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.568480 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.568481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:06.568630 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:06.569217 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:33:06.569227 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:06.570696 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:33:06.572276 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:33:06.572283 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:33:06.572286 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:33:06.572608 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:33:06.572848 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:33:06.572849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:33:06.572849 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:33:06.572989 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:33:06.573441 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:33:06.573448 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:33:06.573618 (+ 170us) client_negotiation.cc:769] Sending connection context
0812 16:33:06.573736 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 16:33:06.573917 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":72,"threads_started":1}
I20260812 16:33:06.574636 32678 heartbeater.cc:347] Connected to a master server at 127.19.238.126:42617
I20260812 16:33:06.574718 32678 heartbeater.cc:464] Registering TS with master...
I20260812 16:33:06.574867 32678 heartbeater.cc:511] Master 127.19.238.126:42617 requested a full tablet report, sending...
I20260812 16:33:06.575243 32162 ts_manager.cc:194] Registered new tserver with Master: 317551a81b9c407c8bfee6a3ddc97c18 (127.19.238.67:39505)
I20260812 16:33:06.575620 32162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:49967
I20260812 16:33:06.584511 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:33:06.592164 32207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.585896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44572 (local address 127.19.238.126:42617)
0812 16:33:06.585950 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:06.585952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:06.586057 (+ 105us) server_negotiation.cc:407] Connection header received
0812 16:33:06.586099 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:06.586099 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:06.586109 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:06.586160 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:06.586772 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.587268 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.588106 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.588288 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.588345 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:06.589620 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:33:06.589622 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:33:06.589622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:33:06.589629 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:33:06.591068 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:06.591441 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:06.591441 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:06.591442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:06.591502 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:06.591672 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:06.591673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:06.591673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:06.591827 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:33:06.591935 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:06.592031 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 16:33:06.592096 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 16:33:06.593923 32162 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:33:06.595556 32162 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:33:06.602881 32690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.598038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:35813 (local address 127.0.0.1:55830)
0812 16:33:06.598261 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:06.598322 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:06.598393 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:06.598743 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:06.598746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:06.598748 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:33:06.598910 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:06.598911 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.600327 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.600328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:06.601651 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:06.601654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.602552 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.602553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:06.602645 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:06.602663 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 16:33:06.602743 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 16:33:06.602796 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":9,"thread_start_us":84,"threads_started":1}
I20260812 16:33:06.602891 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.598179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55830 (local address 127.19.238.65:35813)
0812 16:33:06.598387 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:06.598392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:06.598406 (+ 14us) server_negotiation.cc:407] Connection header received
0812 16:33:06.598535 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:06.598540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:06.598558 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:06.598615 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:33:06.599030 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.600228 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.601746 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.602467 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.602546 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:06.602610 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:06.602771 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 16:33:06.602812 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":103,"threads_started":1}
I20260812 16:33:06.603669 32691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.598123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:34145 (local address 127.0.0.1:50618)
0812 16:33:06.598403 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:06.598414 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:06.598473 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:06.598845 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:06.598845 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:06.598847 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:33:06.598909 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:06.598911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.601468 (+ 2557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.601469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:06.602535 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:06.602538 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.603438 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.603439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:06.603518 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:06.603523 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:33:06.603556 (+ 33us) client_negotiation.cc:240] Negotiation successful
0812 16:33:06.603601 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":103,"threads_started":1}
I20260812 16:33:06.603782 32693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.598273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:39505 (local address 127.0.0.1:46350)
0812 16:33:06.598458 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:06.598514 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:06.598558 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:06.599300 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:06.599300 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:06.599302 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:33:06.599333 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:06.599334 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.600527 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.600528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:06.601651 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:06.601654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.603504 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.603505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:06.603577 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:06.603583 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 16:33:06.603664 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 16:33:06.603711 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":8,"thread_start_us":102,"threads_started":1}
I20260812 16:33:06.603798 32695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.598775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46350 (local address 127.19.238.67:39505)
0812 16:33:06.599136 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:06.599142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:06.599151 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:33:06.599201 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:06.599203 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:06.599218 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:06.599302 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:33:06.599383 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.600410 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.602646 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.603436 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.603503 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:06.603556 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:06.603680 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 16:33:06.603717 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":102,"threads_started":1}
I20260812 16:33:06.604082 32694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.598470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50618 (local address 127.19.238.66:34145)
0812 16:33:06.598664 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:06.598667 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:06.598676 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:33:06.598729 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:06.598731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:06.598748 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:06.598836 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:33:06.600355 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.601359 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.602617 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.603340 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.603861 (+ 521us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:06.603925 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:06.603952 (+ 27us) server_negotiation.cc:299] Negotiation successful
0812 16:33:06.603996 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":100,"threads_started":1}
I20260812 16:33:06.604066 32300 tablet_service.cc:1467] Processing CreateTablet for tablet 15035bdaed4544828d05ff382bce80e4 (DEFAULT_TABLE table=test-table [id=13bffaa5bae4428dad0b83b6084fb57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:33:06.604420 32300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15035bdaed4544828d05ff382bce80e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:33:06.604689 32448 tablet_service.cc:1467] Processing CreateTablet for tablet 15035bdaed4544828d05ff382bce80e4 (DEFAULT_TABLE table=test-table [id=13bffaa5bae4428dad0b83b6084fb57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:33:06.604930 32448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15035bdaed4544828d05ff382bce80e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:33:06.606956 32601 tablet_service.cc:1467] Processing CreateTablet for tablet 15035bdaed4544828d05ff382bce80e4 (DEFAULT_TABLE table=test-table [id=13bffaa5bae4428dad0b83b6084fb57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:33:06.607297 32601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15035bdaed4544828d05ff382bce80e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:33:06.607331 32696 tablet_bootstrap.cc:492] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174: Bootstrap starting.
I20260812 16:33:06.607899 32697 tablet_bootstrap.cc:492] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f: Bootstrap starting.
I20260812 16:33:06.608222 32696 tablet_bootstrap.cc:654] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174: Neither blocks nor log segments found. Creating new log.
I20260812 16:33:06.608598 32697 tablet_bootstrap.cc:654] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f: Neither blocks nor log segments found. Creating new log.
I20260812 16:33:06.608851 32696 log.cc:824] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174: Log is configured to *not* fsync() on all Append() calls
I20260812 16:33:06.609196 32697 log.cc:824] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f: Log is configured to *not* fsync() on all Append() calls
I20260812 16:33:06.609711 32696 tablet_bootstrap.cc:492] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174: No bootstrap required, opened a new log
I20260812 16:33:06.609802 32696 ts_tablet_manager.cc:1397] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:33:06.610355 32697 tablet_bootstrap.cc:492] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f: No bootstrap required, opened a new log
I20260812 16:33:06.610426 32697 ts_tablet_manager.cc:1397] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:33:06.610747 32701 tablet_bootstrap.cc:492] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18: Bootstrap starting.
I20260812 16:33:06.611265 32701 tablet_bootstrap.cc:654] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18: Neither blocks nor log segments found. Creating new log.
I20260812 16:33:06.611770 32701 log.cc:824] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18: Log is configured to *not* fsync() on all Append() calls
I20260812 16:33:06.611900 32696 raft_consensus.cc:348] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53223bf77c14ea0a1deb4ab79b13174" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35813 } } peers { permanent_uuid: "317551a81b9c407c8bfee6a3ddc97c18" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39505 } } peers { permanent_uuid: "fae677c969f14608a287c00990eb584f" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 34145 } }
I20260812 16:33:06.612092 32696 raft_consensus.cc:374] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:33:06.612138 32696 raft_consensus.cc:729] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53223bf77c14ea0a1deb4ab79b13174, State: Initialized, Role: FOLLOWER
I20260812 16:33:06.612298 32696 consensus_queue.cc:260] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53223bf77c14ea0a1deb4ab79b13174" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35813 } } peers { permanent_uuid: "317551a81b9c407c8bfee6a3ddc97c18" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39505 } } peers { permanent_uuid: "fae677c969f14608a287c00990eb584f" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 34145 } }
I20260812 16:33:06.612475 32697 raft_consensus.cc:348] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53223bf77c14ea0a1deb4ab79b13174" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35813 } } peers { permanent_uuid: "317551a81b9c407c8bfee6a3ddc97c18" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39505 } } peers { permanent_uuid: "fae677c969f14608a287c00990eb584f" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 34145 } }
I20260812 16:33:06.612600 32696 ts_tablet_manager.cc:1428] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:33:06.612598 32701 tablet_bootstrap.cc:492] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18: No bootstrap required, opened a new log
I20260812 16:33:06.612629 32697 raft_consensus.cc:374] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:33:06.612671 32701 ts_tablet_manager.cc:1397] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:33:06.612664 32697 raft_consensus.cc:729] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae677c969f14608a287c00990eb584f, State: Initialized, Role: FOLLOWER
I20260812 16:33:06.612720 32373 heartbeater.cc:503] Master 127.19.238.126:42617 was elected leader, sending a full tablet report...
I20260812 16:33:06.612779 32697 consensus_queue.cc:260] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53223bf77c14ea0a1deb4ab79b13174" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35813 } } peers { permanent_uuid: "317551a81b9c407c8bfee6a3ddc97c18" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39505 } } peers { permanent_uuid: "fae677c969f14608a287c00990eb584f" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 34145 } }
I20260812 16:33:06.613080 32697 ts_tablet_manager.cc:1428] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:33:06.613229 32522 heartbeater.cc:503] Master 127.19.238.126:42617 was elected leader, sending a full tablet report...
I20260812 16:33:06.614795 32701 raft_consensus.cc:348] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53223bf77c14ea0a1deb4ab79b13174" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35813 } } peers { permanent_uuid: "317551a81b9c407c8bfee6a3ddc97c18" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39505 } } peers { permanent_uuid: "fae677c969f14608a287c00990eb584f" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 34145 } }
I20260812 16:33:06.614934 32701 raft_consensus.cc:374] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:33:06.614985 32701 raft_consensus.cc:729] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 317551a81b9c407c8bfee6a3ddc97c18, State: Initialized, Role: FOLLOWER
I20260812 16:33:06.615094 32701 consensus_queue.cc:260] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53223bf77c14ea0a1deb4ab79b13174" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35813 } } peers { permanent_uuid: "317551a81b9c407c8bfee6a3ddc97c18" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39505 } } peers { permanent_uuid: "fae677c969f14608a287c00990eb584f" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 34145 } }
I20260812 16:33:06.615345 32701 ts_tablet_manager.cc:1428] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:33:06.615432 32678 heartbeater.cc:503] Master 127.19.238.126:42617 was elected leader, sending a full tablet report...
W20260812 16:33:06.654354 32523 tablet.cc:2369] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:33:06.728021 32703 raft_consensus.cc:482] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:33:06.728191 32703 raft_consensus.cc:504] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53223bf77c14ea0a1deb4ab79b13174" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35813 } } peers { permanent_uuid: "317551a81b9c407c8bfee6a3ddc97c18" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39505 } } peers { permanent_uuid: "fae677c969f14608a287c00990eb584f" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 34145 } }
I20260812 16:33:06.728665 32703 leader_election.cc:288] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 317551a81b9c407c8bfee6a3ddc97c18 (127.19.238.67:39505), fae677c969f14608a287c00990eb584f (127.19.238.66:34145)
I20260812 16:33:06.732873 32694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.728948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:36037 (local address 127.19.238.66:34145)
0812 16:33:06.728994 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:06.728996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:06.729009 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:33:06.729065 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:06.729066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:06.729077 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:06.729138 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:33:06.729391 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.730171 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.731868 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.732468 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.732570 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:06.732581 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:06.732757 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 16:33:06.732800 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 16:33:06.732873 32695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.728941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:51195 (local address 127.19.238.67:39505)
0812 16:33:06.728989 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:06.728991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:06.729078 (+ 87us) server_negotiation.cc:407] Connection header received
0812 16:33:06.729122 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:06.729123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:06.729132 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:06.729190 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:33:06.729391 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.730185 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.731871 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.732468 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.732561 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:06.732572 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:06.732757 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 16:33:06.732800 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:33:06.732879 32377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.728841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:34145 (local address 127.19.238.65:36037)
0812 16:33:06.728918 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:06.728932 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:06.728987 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:06.729160 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:06.729162 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:06.729163 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:33:06.729272 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:06.729274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.730284 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.730285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:06.731722 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:06.731726 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.732580 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.732581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:06.732668 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:06.732674 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 16:33:06.732735 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 16:33:06.732785 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":141}
I20260812 16:33:06.732887 32706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.728862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:39505 (local address 127.19.238.65:51195)
0812 16:33:06.729005 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:06.729015 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:06.729068 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:06.729202 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:06.729202 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:06.729203 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:33:06.729272 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:06.729274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.730284 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.730285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:06.731722 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:06.731726 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.732580 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.732581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:06.732668 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:06.732674 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 16:33:06.732735 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 16:33:06.732781 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":56,"thread_start_us":67,"threads_started":1}
I20260812 16:33:06.733333 32471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15035bdaed4544828d05ff382bce80e4" candidate_uuid: "a53223bf77c14ea0a1deb4ab79b13174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae677c969f14608a287c00990eb584f" is_pre_election: true
I20260812 16:33:06.733529 32471 raft_consensus.cc:2454] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53223bf77c14ea0a1deb4ab79b13174 in term 0.
I20260812 16:33:06.733493 32623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15035bdaed4544828d05ff382bce80e4" candidate_uuid: "a53223bf77c14ea0a1deb4ab79b13174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "317551a81b9c407c8bfee6a3ddc97c18" is_pre_election: true
I20260812 16:33:06.733678 32623 raft_consensus.cc:2454] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53223bf77c14ea0a1deb4ab79b13174 in term 0.
I20260812 16:33:06.733775 32250 leader_election.cc:302] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53223bf77c14ea0a1deb4ab79b13174, fae677c969f14608a287c00990eb584f; no voters:
W20260812 16:33:06.733909 32374 tablet.cc:2369] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:33:06.733922 32703 raft_consensus.cc:2781] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:33:06.733984 32703 raft_consensus.cc:482] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:33:06.734022 32703 raft_consensus.cc:3037] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:33:06.734792 32703 raft_consensus.cc:504] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53223bf77c14ea0a1deb4ab79b13174" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35813 } } peers { permanent_uuid: "317551a81b9c407c8bfee6a3ddc97c18" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39505 } } peers { permanent_uuid: "fae677c969f14608a287c00990eb584f" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 34145 } }
I20260812 16:33:06.734953 32703 leader_election.cc:288] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [CANDIDATE]: Term 1 election: Requested vote from peers 317551a81b9c407c8bfee6a3ddc97c18 (127.19.238.67:39505), fae677c969f14608a287c00990eb584f (127.19.238.66:34145)
I20260812 16:33:06.735121 32623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15035bdaed4544828d05ff382bce80e4" candidate_uuid: "a53223bf77c14ea0a1deb4ab79b13174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "317551a81b9c407c8bfee6a3ddc97c18"
I20260812 16:33:06.735205 32623 raft_consensus.cc:3037] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:33:06.735155 32471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15035bdaed4544828d05ff382bce80e4" candidate_uuid: "a53223bf77c14ea0a1deb4ab79b13174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae677c969f14608a287c00990eb584f"
I20260812 16:33:06.735294 32471 raft_consensus.cc:3037] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:33:06.736129 32623 raft_consensus.cc:2454] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53223bf77c14ea0a1deb4ab79b13174 in term 1.
I20260812 16:33:06.736240 32471 raft_consensus.cc:2454] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53223bf77c14ea0a1deb4ab79b13174 in term 1.
I20260812 16:33:06.736337 32249 leader_election.cc:302] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 317551a81b9c407c8bfee6a3ddc97c18, a53223bf77c14ea0a1deb4ab79b13174; no voters:
I20260812 16:33:06.736446 32703 raft_consensus.cc:2781] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:33:06.736593 32703 raft_consensus.cc:686] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [term 1 LEADER]: Becoming Leader. State: Replica: a53223bf77c14ea0a1deb4ab79b13174, State: Running, Role: LEADER
I20260812 16:33:06.736694 32703 consensus_queue.cc:237] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53223bf77c14ea0a1deb4ab79b13174" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35813 } } peers { permanent_uuid: "317551a81b9c407c8bfee6a3ddc97c18" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39505 } } peers { permanent_uuid: "fae677c969f14608a287c00990eb584f" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 34145 } }
I20260812 16:33:06.737665 32160 catalog_manager.cc:5495] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53223bf77c14ea0a1deb4ab79b13174 (127.19.238.65). New cstate: current_term: 1 leader_uuid: "a53223bf77c14ea0a1deb4ab79b13174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53223bf77c14ea0a1deb4ab79b13174" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "317551a81b9c407c8bfee6a3ddc97c18" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae677c969f14608a287c00990eb584f" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 34145 } health_report { overall_health: UNKNOWN } } }
I20260812 16:33:06.747769 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.745272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55842 (local address 127.19.238.65:35813)
0812 16:33:06.745337 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:06.745339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:06.745348 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:33:06.745390 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:06.745391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:06.745419 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:06.745499 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:33:06.745667 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.746162 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.746926 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:06.747146 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:06.747224 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:06.747403 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:33:06.747516 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:33:06.747616 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:06.747636 (+ 20us) server_negotiation.cc:299] Negotiation successful
0812 16:33:06.747658 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 16:33:06.749614 32471 raft_consensus.cc:1261] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f [term 1 FOLLOWER]: Refusing update from remote peer a53223bf77c14ea0a1deb4ab79b13174: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:33:06.749620 32623 raft_consensus.cc:1261] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18 [term 1 FOLLOWER]: Refusing update from remote peer a53223bf77c14ea0a1deb4ab79b13174: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:33:06.749996 32703 consensus_queue.cc:1035] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae677c969f14608a287c00990eb584f" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 34145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:33:06.750166 32707 consensus_queue.cc:1035] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "317551a81b9c407c8bfee6a3ddc97c18" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:33:06.753877 32711 mvcc.cc:205] Tried to move back new op lower bound from 7317718576123600896 to 7317718576074182656. Current Snapshot: MvccSnapshot[applied={T|T < 7317718576123600896 or (T in {7317718576123600896})}]
I20260812 16:33:06.754057 32713 mvcc.cc:205] Tried to move back new op lower bound from 7317718576123600896 to 7317718576074182656. Current Snapshot: MvccSnapshot[applied={T|T < 7317718576123600896 or (T in {7317718576123600896})}]
I20260812 16:33:06.755228 32162 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:33:06.755337 32162 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:33:06.756165 32162 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9de6bb506ef4408bd53a030d79f0010: Sending DeleteTablet for 3 replicas of tablet 15035bdaed4544828d05ff382bce80e4
I20260812 16:33:06.756510 32448 tablet_service.cc:1514] Processing DeleteTablet for tablet 15035bdaed4544828d05ff382bce80e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:33:06 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:50618
I20260812 16:33:06.756590 32300 tablet_service.cc:1514] Processing DeleteTablet for tablet 15035bdaed4544828d05ff382bce80e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:33:06 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:55830
I20260812 16:33:06.756610 32601 tablet_service.cc:1514] Processing DeleteTablet for tablet 15035bdaed4544828d05ff382bce80e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:33:06 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:46350
I20260812 16:33:06.756825 32723 tablet_replica.cc:330] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18: stopping tablet replica
I20260812 16:33:06.756853 32724 tablet_replica.cc:330] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174: stopping tablet replica
I20260812 16:33:06.756925 32723 raft_consensus.cc:2229] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:33:06.756947 32724 raft_consensus.cc:2229] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:33:06.757031 32723 raft_consensus.cc:2258] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:33:06.757079 32724 raft_consensus.cc:2258] T 15035bdaed4544828d05ff382bce80e4 P a53223bf77c14ea0a1deb4ab79b13174 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:33:06.757306 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 32230
I20260812 16:33:06.757334 32722 tablet_replica.cc:330] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f: stopping tablet replica
I20260812 16:33:06.757436 32722 raft_consensus.cc:2229] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:33:06.757537 32722 raft_consensus.cc:2258] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:33:06.757546 32723 ts_tablet_manager.cc:1905] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:33:06.758060 32722 ts_tablet_manager.cc:1905] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:33:06.759582 32723 ts_tablet_manager.cc:1918] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:33:06.759652 32723 log.cc:1196] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/15035bdaed4544828d05ff382bce80e4
I20260812 16:33:06.759948 32723 ts_tablet_manager.cc:1939] T 15035bdaed4544828d05ff382bce80e4 P 317551a81b9c407c8bfee6a3ddc97c18: Deleting consensus metadata
I20260812 16:33:06.760221 32722 ts_tablet_manager.cc:1918] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:33:06.760418 32722 log.cc:1196] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/15035bdaed4544828d05ff382bce80e4
I20260812 16:33:06.760382 32144 catalog_manager.cc:4841] TS 317551a81b9c407c8bfee6a3ddc97c18 (127.19.238.67:39505): tablet 15035bdaed4544828d05ff382bce80e4 (table test-table [id=13bffaa5bae4428dad0b83b6084fb57f]) successfully deleted
I20260812 16:33:06.760731 32722 ts_tablet_manager.cc:1939] T 15035bdaed4544828d05ff382bce80e4 P fae677c969f14608a287c00990eb584f: Deleting consensus metadata
I20260812 16:33:06.761317 32145 catalog_manager.cc:4841] TS fae677c969f14608a287c00990eb584f (127.19.238.66:34145): tablet 15035bdaed4544828d05ff382bce80e4 (table test-table [id=13bffaa5bae4428dad0b83b6084fb57f]) successfully deleted
W20260812 16:33:06.766940 32144 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.238.65:35813 (error 108)
I20260812 16:33:06.767163 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 32380
I20260812 16:33:06.768213 32690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:06.768030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:35813 (local address 127.0.0.1:55844)
0812 16:33:06.768109 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:06.768163 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 16:33:06.768509 32144 catalog_manager.cc:4584] TS a53223bf77c14ea0a1deb4ab79b13174 (127.19.238.65:35813): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 15035bdaed4544828d05ff382bce80e4: Network error: Client connection negotiation failed: client connection to 127.19.238.65:35813: connect: Connection refused (error 111)
I20260812 16:33:06.775766 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 32527
I20260812 16:33:06.781553 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 32125
2026-08-12T16:33:06Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2154 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:33:06 dist-test-slave-mvvj krb5kdc[32730](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:33:06 dist-test-slave-mvvj krb5kdc[32730](info): set up 2 sockets
Aug 12 16:33:06 dist-test-slave-mvvj krb5kdc[32730](info): commencing operation
krb5kdc: starting...
W20260812 16:33:07.924015 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:33:08 dist-test-slave-mvvj krb5kdc[32730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:33:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:33:08Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.19.238.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.19.238.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.19.238.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/rangeradmin_127.19.238.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.19.238.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/rangeradmin_127.19.238.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.19.238.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.19.238.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.19.238.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/rangerlookup_127.19.238.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.19.238.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/rangerlookup_127.19.238.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.238.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.19.238.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.238.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/HTTP_127.19.238.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.238.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/HTTP_127.19.238.84@KRBTEST.COM.keytab.
I20260812 16:33:08.110798 20409 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskt79009/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/postgres -l logfile start
2026-08-12 16:33:10.896 UTC [32764] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 16:33:10.896 UTC [32764] LOG: listening on IPv4 address "127.19.238.84", port 54971
2026-08-12 16:33:10.899 UTC [32764] LOG: listening on Unix socket "/tmp/.s.PGSQL.54971"
2026-08-12 16:33:10.922 UTC [32767] LOG: database system was shut down at 2026-08-12 16:33:09 UTC
2026-08-12 16:33:10.926 UTC [32764] LOG: database system is ready to accept connections
I20260812 16:33:11.944749 20409 mini_postgres.cc:96] Postgres bound to 54971
2026-08-12 16:33:11.952 UTC [308] FATAL: database "slave" does not exist
127.19.238.84:54971 - accepting connections
I20260812 16:33:11.953523 20409 mini_ranger.cc:161] Starting Ranger...
I20260812 16:33:11.970486 20409 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 16:33:12.541503 20409 mini_ranger.cc:87] Created ranger Postgres database
I20260812 16:33:12.541610 20409 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ranger-admin
2026-08-12 16:33:12,910 [I] DB FLAVOR :POSTGRES
2026-08-12 16:33:12,910 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 16:33:12,910 [I] Checking connection..
2026-08-12 16:33:12,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 16:33:13,192 [I] Checking connection passed.
2026-08-12 16:33:13,193 [I] --------- Verifying version history table ---------
2026-08-12 16:33:13,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 16:33:13,436 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 16:33:13,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 16:33:13,699 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 16:33:13,700 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 16:33:13,700 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 16:33:13.957 UTC [382] WARNING: there is no transaction in progress
2026-08-12 16:33:13,965 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 16:33:13,965 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 16:33:14,218 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 16:33:14,218 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 16:33:14,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 16:33:14,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 16:33:14,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-mvvj.c.gcp-upstream.internal','N') ;"
2026-08-12 16:33:15,010 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 16:33:15,253 [I] Table x_portal_user does not exist in database ranger
2026-08-12 16:33:15,253 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 16:33:15,518 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 16:33:15,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 16:33:15,763 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 16:33:16,007 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 16:33:16,007 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 16:33:17.141 UTC [505] WARNING: there is no transaction in progress
2026-08-12 16:33:17.160 UTC [505] WARNING: there is no transaction in progress
2026-08-12 16:33:17.180 UTC [505] WARNING: there is no transaction in progress
2026-08-12 16:33:17.198 UTC [505] WARNING: there is no transaction in progress
2026-08-12 16:33:17.217 UTC [505] WARNING: there is no transaction in progress
2026-08-12 16:33:17.239 UTC [505] WARNING: there is no transaction in progress
2026-08-12 16:33:17.306 UTC [505] WARNING: there is no transaction in progress
2026-08-12 16:33:17.315 UTC [505] WARNING: there is no transaction in progress
2026-08-12 16:33:17.334 UTC [505] WARNING: there is no transaction in progress
2026-08-12 16:33:17.346 UTC [505] WARNING: there is no transaction in progress
2026-08-12 16:33:17.359 UTC [505] WARNING: there is no transaction in progress
2026-08-12 16:33:17.378 UTC [505] WARNING: there is no transaction in progress
2026-08-12 16:33:17.390 UTC [505] WARNING: there is no transaction in progress
2026-08-12 16:33:17.398 UTC [505] WARNING: there is no transaction in progress
2026-08-12 16:33:17.407 UTC [505] WARNING: there is no transaction in progress
2026-08-12 16:33:17.415 UTC [505] WARNING: there is no transaction in progress
2026-08-12 16:33:18,269 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 16:33:18,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 16:33:18,530 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 16:33:18,530 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 16:33:18,788 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 16:33:18,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 16:33:19,036 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 16:33:19,036 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 16:33:19,302 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 16:33:19,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-mvvj.c.gcp-upstream.internal';"
2026-08-12 16:33:19,840 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 16:33:19,840 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 16:33:20,105 [I] DB_PATCHES have already been applied
I20260812 16:33:20.112136 20409 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ranger-admin:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskt79009/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 16:33:20.112213 20409 mini_ranger.cc:193] Using host: 127.19.238.84
I20260812 16:33:20.116384 20409 mini_ranger.cc:227] Ranger admin URL: http://127.19.238.84:52435
Aug 12, 2026 4:33:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 4:33:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 4:33:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 4:33:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 4:33:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 4:33:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 4:33:21 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 16:33:22.489310 20409 mini_ranger.cc:160] Time spent starting Ranger: real 10.536s user 0.001s sys 0.014s
Aug 12, 2026 4:33:43 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 4:33:43 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 12, 2026 4:33:43 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 4:33:43 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 4:33:44 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 16:33:44.922850 20409 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.19.238.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.19.238.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.19.238.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/rangerkms_127.19.238.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.19.238.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/rangerkms_127.19.238.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.238.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.19.238.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.238.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/HTTP_127.19.238.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.238.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/HTTP_127.19.238.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 16:33:45 dist-test-slave-mvvj krb5kdc[32730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552425, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 16:33:45.021508 20409 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 16:33:45.032575 20409 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 16:33:45.978399 20409 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 16:33:45.978472 20409 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ranger-kms
I20260812 16:33:45.978485 20409 mini_ranger_kms.cc:227] Using postgres at 127.19.238.84:54971
2026-08-12 16:33:46,240 [I] DB FLAVOR :POSTGRES
2026-08-12 16:33:46,241 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 16:33:46,241 [I] Checking connection
2026-08-12 16:33:46,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 16:33:46,495 [I] connection success
2026-08-12 16:33:46,495 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 16:33:46,495 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 16:33:46,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 16:33:46,770 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 16:33:46,771 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 16:33:46,771 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 16:33:46,771 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:54971/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 16:33:47,064 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 16:33:47.596706 20409 mini_ranger_kms.cc:310] Created kms service
I20260812 16:33:47.783402 20409 mini_ranger_kms.cc:326] Created kudu user
I20260812 16:33:47.837524 20409 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 16:33:47 dist-test-slave-mvvj krb5kdc[32730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552425, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
I20260812 16:33:48.291033 20409 mini_ranger_kms.cc:384] Added ranger policy
I20260812 16:33:48.291168 20409 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ranger-kms:/tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 16:33:48.291191 20409 mini_ranger_kms.cc:240] Using host: 127.19.238.84
I20260812 16:33:48.296017 20409 mini_ranger_kms.cc:276] Ranger KMS PID: 712
I20260812 16:33:48.296168 20409 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.19.238.84:53571
Aug 12, 2026 4:33:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 4:33:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 4:33:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 4:33:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 4:33:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 4:33:49 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260812 16:33:50.852113 20409 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.831s user 0.000s sys 0.215s
I20260812 16:33:50.852222 20409 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 16:33:50.852303 20409 mini_ranger_kms.cc:401] 127.19.238.84:53571/kms/v1/keys
Aug 12 16:33:50 dist-test-slave-mvvj krb5kdc[32730](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786552430, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.238.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 16:33:51 dist-test-slave-mvvj krb5kdc[32730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552430, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.238.84@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
Aug 12 16:33:51 dist-test-slave-mvvj krb5kdc[32730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552430, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.238.84@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
Aug 12 16:33:54 dist-test-slave-mvvj krb5kdc[32730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:33:54.119467 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:42439
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:45779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:42439
--ranger_config_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.238.84:53571/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:33:54.206602 774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:33:54.206835 774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:33:54.206859 774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:33:54.208189 774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:33:54.208236 774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:33:54.208251 774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:33:54.208264 774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:33:54.208276 774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:33:54.210034 774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:45779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/master-0/wal
--ranger_kms_url=127.19.238.84:53571/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:42439
--ranger_config_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:42439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:33:54.210300 774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:33:54.210546 774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:33:54.216150 782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:54.216267 781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:54.216199 784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:33:54.216274 774 server_base.cc:1034] running on GCE node
I20260812 16:33:54.216688 774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:33:54.217057 774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:33:54.218250 774 hybrid_clock.cc:648] HybridClock initialized: now 1786552434218235 us; error 30 us; skew 500 ppm
Aug 12 16:33:54 dist-test-slave-mvvj krb5kdc[32730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552434, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:33:54.225507 774 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:33:54.226073 774 webserver.cc:460] Webserver started at http://127.19.238.126:33083/ using document root <none> and password file <none>
I20260812 16:33:54.226332 774 fs_manager.cc:359] Metadata directory not provided
I20260812 16:33:54.226408 774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:33:54.226562 774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:33:54 dist-test-slave-mvvj krb5kdc[32730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552434, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
I20260812 16:33:54.293989 774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "0ca3b42d7b284b828fd7a755652ad3d8"
format_stamp: "Formatted at 2026-08-12 16:33:54 on dist-test-slave-mvvj"
server_key: "53760850c4757ceb30c6a5725f83c9d4"
server_key_iv: "caaa3adb7022ade26a35f8d7ff540391"
server_key_version: "kuduclusterkey@0"
I20260812 16:33:54.294517 774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "0ca3b42d7b284b828fd7a755652ad3d8"
format_stamp: "Formatted at 2026-08-12 16:33:54 on dist-test-slave-mvvj"
server_key: "53760850c4757ceb30c6a5725f83c9d4"
server_key_iv: "caaa3adb7022ade26a35f8d7ff540391"
server_key_version: "kuduclusterkey@0"
I20260812 16:33:54.296510 774 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 16:33:54.312575 794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:54.312819 774 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:33:54.312924 774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "0ca3b42d7b284b828fd7a755652ad3d8"
format_stamp: "Formatted at 2026-08-12 16:33:54 on dist-test-slave-mvvj"
server_key: "53760850c4757ceb30c6a5725f83c9d4"
server_key_iv: "caaa3adb7022ade26a35f8d7ff540391"
server_key_version: "kuduclusterkey@0"
I20260812 16:33:54.313040 774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:33:54.327540 774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:33:54.329890 774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:33:54.330058 774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:33:54.335268 774 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:42439
I20260812 16:33:54.335285 850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:42439 every 8 connection(s)
I20260812 16:33:54.335820 774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:33:54.336615 851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:33:54.336987 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 774
I20260812 16:33:54.337093 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:33:54.337296 20409 external_mini_cluster.cc:1428] Setting key 795c227aee5f56c11aec8f5875a9e3fe
I20260812 16:33:54.339697 851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8: Bootstrap starting.
I20260812 16:33:54.340492 851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8: Neither blocks nor log segments found. Creating new log.
I20260812 16:33:54.340794 851 log.cc:824] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8: Log is configured to *not* fsync() on all Append() calls
I20260812 16:33:54.341616 851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8: No bootstrap required, opened a new log
I20260812 16:33:54.344347 851 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca3b42d7b284b828fd7a755652ad3d8" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42439 } }
I20260812 16:33:54.344553 851 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:33:54.344595 851 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ca3b42d7b284b828fd7a755652ad3d8, State: Initialized, Role: FOLLOWER
Aug 12 16:33:54 dist-test-slave-mvvj krb5kdc[32730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:33:54.344794 851 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca3b42d7b284b828fd7a755652ad3d8" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42439 } }
I20260812 16:33:54.344893 851 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:33:54.344925 851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:33:54.344995 851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:33:54.345765 851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca3b42d7b284b828fd7a755652ad3d8" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42439 } }
I20260812 16:33:54.345898 851 leader_election.cc:302] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ca3b42d7b284b828fd7a755652ad3d8; no voters:
I20260812 16:33:54.346092 851 leader_election.cc:288] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:33:54.346171 856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:33:54.346356 851 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:33:54.346791 856 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8 [term 1 LEADER]: Becoming Leader. State: Replica: 0ca3b42d7b284b828fd7a755652ad3d8, State: Running, Role: LEADER
I20260812 16:33:54.346958 856 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca3b42d7b284b828fd7a755652ad3d8" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42439 } }
I20260812 16:33:54.347381 856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ca3b42d7b284b828fd7a755652ad3d8. Latest consensus state: current_term: 1 leader_uuid: "0ca3b42d7b284b828fd7a755652ad3d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca3b42d7b284b828fd7a755652ad3d8" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42439 } } }
I20260812 16:33:54.347535 856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8 [sys.catalog]: This master's current role is: LEADER
I20260812 16:33:54.348248 857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ca3b42d7b284b828fd7a755652ad3d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca3b42d7b284b828fd7a755652ad3d8" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42439 } } }
I20260812 16:33:54.348423 857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8 [sys.catalog]: This master's current role is: LEADER
I20260812 16:33:54.348520 851 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 16:33:54.350914 854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:54.337918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38278 (local address 127.19.238.126:42439)
0812 16:33:54.338126 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:54.338150 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:54.338213 (+ 63us) server_negotiation.cc:407] Connection header received
0812 16:33:54.338788 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:54.338798 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:54.338883 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:54.339050 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:54.339903 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:54.341309 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:54.342121 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:54.342564 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:54.342693 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:54.345164 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:33:54.345168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:33:54.345168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:33:54.345179 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:33:54.347272 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:54.347726 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:54.347729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:54.347729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:54.347897 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:54.348122 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:54.348123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:54.348124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:54.348454 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:33:54.348607 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:54.348899 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 16:33:54.349040 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":87,"threads_started":1}
I20260812 16:33:54.355176 866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:33:54.355789 866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:33:54.357766 866 catalog_manager.cc:1347] Generated new cluster ID: 604a9f9b342743aea274663bf692c895
I20260812 16:33:54.357815 866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:33:54.370648 866 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:33:54.371632 866 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:33:54.380158 866 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8: Generated new TSK 0
I20260812 16:33:54.380404 866 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
I20260812 16:33:55.127290 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.238.84:53571/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42439
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:45779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:33:55.254703 898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:33:55.255024 898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:33:55.255074 898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:33:55.256915 898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:33:55.256982 898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:33:55.257026 898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:33:55.259266 898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:45779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.19.238.84:53571/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:33:55.259553 898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:33:55.259932 898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 16:33:55.266286 898 server_base.cc:1034] running on GCE node
W20260812 16:33:55.266485 906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:55.266492 904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:55.266644 903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:33:55.266999 898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:33:55.267307 898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:33:55.268486 898 hybrid_clock.cc:648] HybridClock initialized: now 1786552435268468 us; error 35 us; skew 500 ppm
Aug 12 16:33:55 dist-test-slave-mvvj krb5kdc[32730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552435, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:33:55.271721 898 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:33:55.272215 898 webserver.cc:460] Webserver started at http://127.19.238.65:43317/ using document root <none> and password file <none>
I20260812 16:33:55.272451 898 fs_manager.cc:359] Metadata directory not provided
I20260812 16:33:55.272521 898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:33:55.272671 898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:33:55 dist-test-slave-mvvj krb5kdc[32730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552435, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
I20260812 16:33:55.287127 898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "41bd6873229a4825b3d102ac8ae020cd"
format_stamp: "Formatted at 2026-08-12 16:33:55 on dist-test-slave-mvvj"
server_key: "c2b7a9a66abeb13a84fa72b2b5ec8ece"
server_key_iv: "89a59df93ef3f83363ac22989d504de1"
server_key_version: "kuduclusterkey@0"
I20260812 16:33:55.287706 898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "41bd6873229a4825b3d102ac8ae020cd"
format_stamp: "Formatted at 2026-08-12 16:33:55 on dist-test-slave-mvvj"
server_key: "c2b7a9a66abeb13a84fa72b2b5ec8ece"
server_key_iv: "89a59df93ef3f83363ac22989d504de1"
server_key_version: "kuduclusterkey@0"
I20260812 16:33:55.289657 898 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260812 16:33:55.306957 915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:55.307227 898 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260812 16:33:55.307360 898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "41bd6873229a4825b3d102ac8ae020cd"
format_stamp: "Formatted at 2026-08-12 16:33:55 on dist-test-slave-mvvj"
server_key: "c2b7a9a66abeb13a84fa72b2b5ec8ece"
server_key_iv: "89a59df93ef3f83363ac22989d504de1"
server_key_version: "kuduclusterkey@0"
I20260812 16:33:55.307487 898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:33:55.336408 898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:33:55.338769 898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:33:55.338941 898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:33:55.339231 898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:33:55.339649 898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:33:55.339709 898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:55.339747 898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:33:55.339766 898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:55.347484 898 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:36307
I20260812 16:33:55.347568 1050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:36307 every 8 connection(s)
I20260812 16:33:55.348034 898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
Aug 12 16:33:55 dist-test-slave-mvvj krb5kdc[32730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552435, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:33:55.357105 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 898
I20260812 16:33:55.357221 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:33:55.357431 20409 external_mini_cluster.cc:1428] Setting key e89d838c40949b10aed058989fc6a4e4
I20260812 16:33:55.360528 1053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:55.348823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:42439 (local address 127.19.238.65:48451)
0812 16:33:55.349124 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:55.349169 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:55.349851 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:55.350183 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:55.350187 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:55.350396 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:55.350603 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:55.350607 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.351749 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.351750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:55.352801 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:55.352804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.353436 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.353437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:55.353589 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:55.354284 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:33:55.354294 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:55.356158 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:33:55.358615 (+ 2457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:33:55.358624 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:33:55.358627 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:33:55.358895 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:33:55.359160 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:33:55.359162 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:33:55.359162 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:33:55.359325 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:33:55.359686 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:33:55.359695 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:33:55.359926 (+ 231us) client_negotiation.cc:769] Sending connection context
0812 16:33:55.360066 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 16:33:55.360254 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":99,"threads_started":1}
I20260812 16:33:55.361006 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:55.348884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:48451 (local address 127.19.238.126:42439)
0812 16:33:55.349908 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:55.349911 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:55.349958 (+ 47us) server_negotiation.cc:407] Connection header received
0812 16:33:55.349989 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:55.349991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:55.350007 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:55.350101 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:55.350716 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.351625 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.352905 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.353250 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.353348 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:55.356291 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:33:55.356293 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:33:55.356294 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:33:55.356302 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:33:55.358522 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:55.359003 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:55.359004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:55.359004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:55.359078 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:55.359416 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:55.359417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:55.359418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:55.359584 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:33:55.359654 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:55.360778 (+ 1124us) server_negotiation.cc:299] Negotiation successful
0812 16:33:55.360881 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":992,"thread_start_us":115,"threads_started":1}
I20260812 16:33:55.362722 1051 heartbeater.cc:347] Connected to a master server at 127.19.238.126:42439
I20260812 16:33:55.362846 1051 heartbeater.cc:464] Registering TS with master...
I20260812 16:33:55.363052 1051 heartbeater.cc:511] Master 127.19.238.126:42439 requested a full tablet report, sending...
I20260812 16:33:55.363648 813 ts_manager.cc:194] Registered new tserver with Master: 41bd6873229a4825b3d102ac8ae020cd (127.19.238.65:36307)
I20260812 16:33:55.364639 813 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:48451
WARNING: no policy specified for kudu/127.19.238.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
I20260812 16:33:55.393010 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.238.84:53571/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42439
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:45779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:33:55 dist-test-slave-mvvj krb5kdc[32730](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786552435, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 16:33:55.523818 1057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:33:55.524125 1057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:33:55.524166 1057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:33:55.525977 1057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:33:55.526080 1057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:33:55.526139 1057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:33:55.528400 1057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:45779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.19.238.84:53571/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:33:55.528824 1057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:33:55.529132 1057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:33:55.535063 1068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:55.535063 1066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:55.535625 1065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:33:55.537209 1057 server_base.cc:1034] running on GCE node
I20260812 16:33:55.537569 1057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:33:55.537859 1057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:33:55.539036 1057 hybrid_clock.cc:648] HybridClock initialized: now 1786552435539016 us; error 35 us; skew 500 ppm
Aug 12 16:33:55 dist-test-slave-mvvj krb5kdc[32730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552435, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:33:55.541903 1057 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:33:55.542438 1057 webserver.cc:460] Webserver started at http://127.19.238.66:41361/ using document root <none> and password file <none>
I20260812 16:33:55.542735 1057 fs_manager.cc:359] Metadata directory not provided
I20260812 16:33:55.542806 1057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:33:55.542954 1057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:33:55 dist-test-slave-mvvj krb5kdc[32730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552435, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
I20260812 16:33:55.562601 1057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "7c3fbee1e1994b4e9917e7abd14ba357"
format_stamp: "Formatted at 2026-08-12 16:33:55 on dist-test-slave-mvvj"
server_key: "1147d995b03e935da3125520cd5d0a8a"
server_key_iv: "0e148104fe1bb589daf68ce59800efd8"
server_key_version: "kuduclusterkey@0"
I20260812 16:33:55.563073 1057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "7c3fbee1e1994b4e9917e7abd14ba357"
format_stamp: "Formatted at 2026-08-12 16:33:55 on dist-test-slave-mvvj"
server_key: "1147d995b03e935da3125520cd5d0a8a"
server_key_iv: "0e148104fe1bb589daf68ce59800efd8"
server_key_version: "kuduclusterkey@0"
I20260812 16:33:55.565054 1057 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260812 16:33:55.580515 1076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:55.580806 1057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 16:33:55.580907 1057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "7c3fbee1e1994b4e9917e7abd14ba357"
format_stamp: "Formatted at 2026-08-12 16:33:55 on dist-test-slave-mvvj"
server_key: "1147d995b03e935da3125520cd5d0a8a"
server_key_iv: "0e148104fe1bb589daf68ce59800efd8"
server_key_version: "kuduclusterkey@0"
I20260812 16:33:55.581014 1057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:33:55.607192 1057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:33:55.609818 1057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:33:55.610046 1057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:33:55.610387 1057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:33:55.610805 1057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:33:55.610898 1057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:55.610973 1057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:33:55.611034 1057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:55.618403 1057 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:33159
I20260812 16:33:55.618942 1057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:33:55.620355 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 1057
I20260812 16:33:55.620484 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:33:55.620741 20409 external_mini_cluster.cc:1428] Setting key 3b6df3bf9a14b97789387f0ae77720a0
I20260812 16:33:55.620981 1205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:33159 every 8 connection(s)
WARNING: no policy specified for kudu/127.19.238.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.67@KRBTEST.COM" created.
Aug 12 16:33:55 dist-test-slave-mvvj krb5kdc[32730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552435, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:33:55.645963 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:55.623970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:50055 (local address 127.19.238.126:42439)
0812 16:33:55.624023 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:55.624025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:55.625915 (+ 1890us) server_negotiation.cc:407] Connection header received
0812 16:33:55.626685 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:55.626687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:55.626706 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:55.626737 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:55.627640 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.628413 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.629906 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.630244 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.630294 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:55.642160 (+ 11866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:33:55.642163 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:33:55.642163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:33:55.642173 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:33:55.644112 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:55.644709 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:55.644710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:55.644711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:55.644776 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:55.645130 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:55.645131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:55.645131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:55.645287 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:33:55.645341 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:55.645743 (+ 402us) server_negotiation.cc:299] Negotiation successful
0812 16:33:55.645847 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 16:33:55.646644 1210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:55.623806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:42439 (local address 127.19.238.66:50055)
0812 16:33:55.625754 (+ 1948us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:55.625812 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:55.626546 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:55.626968 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:55.626973 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:55.627237 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:55.627524 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:55.627531 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.628542 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.628542 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:55.629798 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:55.629801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.630399 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.630399 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:55.630549 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:55.640053 (+ 9504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:33:55.640069 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:55.641975 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:33:55.644240 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:33:55.644252 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:33:55.644256 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:33:55.644608 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:33:55.644856 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:33:55.644857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:33:55.644857 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:33:55.645039 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:33:55.645391 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:33:55.645398 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:33:55.645582 (+ 184us) client_negotiation.cc:769] Sending connection context
0812 16:33:55.646122 (+ 540us) client_negotiation.cc:240] Negotiation successful
0812 16:33:55.646352 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1892,"thread_start_us":291,"threads_started":1}
I20260812 16:33:55.647459 1206 heartbeater.cc:347] Connected to a master server at 127.19.238.126:42439
I20260812 16:33:55.647567 1206 heartbeater.cc:464] Registering TS with master...
I20260812 16:33:55.647779 1206 heartbeater.cc:511] Master 127.19.238.126:42439 requested a full tablet report, sending...
I20260812 16:33:55.648291 813 ts_manager.cc:194] Registered new tserver with Master: 7c3fbee1e1994b4e9917e7abd14ba357 (127.19.238.66:33159)
I20260812 16:33:55.648841 813 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:50055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
I20260812 16:33:55.661504 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.238.84:53571/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42439
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:45779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:33:55.785920 1212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:33:55.786319 1212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:33:55.786414 1212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:33:55.788231 1212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:33:55.788296 1212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:33:55.788338 1212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:33:55.790573 1212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:45779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.19.238.84:53571/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:33:55.790856 1212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:33:55.791196 1212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:33:55.797585 1222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:55.798528 1219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:33:55.798806 1217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:33:55.799197 1212 server_base.cc:1034] running on GCE node
I20260812 16:33:55.799443 1212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:33:55.799707 1212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:33:55.800885 1212 hybrid_clock.cc:648] HybridClock initialized: now 1786552435800851 us; error 46 us; skew 500 ppm
Aug 12 16:33:55 dist-test-slave-mvvj krb5kdc[32730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552435, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:33:55.804306 1212 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:33:55.804940 1212 webserver.cc:460] Webserver started at http://127.19.238.67:36651/ using document root <none> and password file <none>
I20260812 16:33:55.805241 1212 fs_manager.cc:359] Metadata directory not provided
I20260812 16:33:55.805320 1212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:33:55.805481 1212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:33:55 dist-test-slave-mvvj krb5kdc[32730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552435, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
I20260812 16:33:55.823210 1212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "5816e8856725446a9825f22b88b1d46e"
format_stamp: "Formatted at 2026-08-12 16:33:55 on dist-test-slave-mvvj"
server_key: "bb3f15a7a7f0821a3f66b98bbfa786e9"
server_key_iv: "9f0322e62c226e781900a3aab73e8a42"
server_key_version: "kuduclusterkey@0"
I20260812 16:33:55.823648 1212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "5816e8856725446a9825f22b88b1d46e"
format_stamp: "Formatted at 2026-08-12 16:33:55 on dist-test-slave-mvvj"
server_key: "bb3f15a7a7f0821a3f66b98bbfa786e9"
server_key_iv: "9f0322e62c226e781900a3aab73e8a42"
server_key_version: "kuduclusterkey@0"
I20260812 16:33:55.825250 1212 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:33:55.838410 1229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:55.838650 1212 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:33:55.838755 1212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "5816e8856725446a9825f22b88b1d46e"
format_stamp: "Formatted at 2026-08-12 16:33:55 on dist-test-slave-mvvj"
server_key: "bb3f15a7a7f0821a3f66b98bbfa786e9"
server_key_iv: "9f0322e62c226e781900a3aab73e8a42"
server_key_version: "kuduclusterkey@0"
I20260812 16:33:55.838881 1212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:33:55.851941 1212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:33:55.854285 1212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:33:55.854568 1212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:33:55.854878 1212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:33:55.855296 1212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:33:55.855360 1212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:55.855438 1212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:33:55.855471 1212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:33:55.873723 1212 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:33001
I20260812 16:33:55.874241 1212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:33:55.878335 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 1212
I20260812 16:33:55.878469 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:33:55.878676 20409 external_mini_cluster.cc:1428] Setting key 91153f8d8ddaa830154c93a1958dacc3
I20260812 16:33:55.887621 1356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:33001 every 8 connection(s)
Aug 12 16:33:55 dist-test-slave-mvvj krb5kdc[32730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552435, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:33:55.896560 1359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:55.875160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:42439 (local address 127.19.238.67:47651)
0812 16:33:55.875722 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:55.875768 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:55.877096 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:55.877480 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:55.877484 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:55.877736 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:33:55.878013 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:55.878019 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.879751 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.879752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:55.881027 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:55.881031 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.881582 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.881583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:55.881749 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:55.890385 (+ 8636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:33:55.890399 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:33:55.892220 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:33:55.894559 (+ 2339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:33:55.894569 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:33:55.894572 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:33:55.894907 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:33:55.895191 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:33:55.895192 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:33:55.895193 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:33:55.895359 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:33:55.895702 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:33:55.895710 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:33:55.895958 (+ 248us) client_negotiation.cc:769] Sending connection context
0812 16:33:55.896084 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 16:33:55.896278 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":84,"threads_started":1}
I20260812 16:33:55.897279 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:55.875464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:47651 (local address 127.19.238.126:42439)
0812 16:33:55.875513 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:55.875515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:55.875895 (+ 380us) server_negotiation.cc:407] Connection header received
0812 16:33:55.877234 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:55.877235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:55.877248 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:55.877281 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:55.878126 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.878919 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.881137 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.881442 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.881484 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:55.892365 (+ 10881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:33:55.892367 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:33:55.892367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:33:55.892378 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:33:55.894457 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:55.895023 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:55.895024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:55.895025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:55.895098 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:55.895445 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:55.895446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:55.895446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:55.895602 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:33:55.895669 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:55.897078 (+ 1409us) server_negotiation.cc:299] Negotiation successful
0812 16:33:55.897180 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 16:33:55.898167 1357 heartbeater.cc:347] Connected to a master server at 127.19.238.126:42439
I20260812 16:33:55.898278 1357 heartbeater.cc:464] Registering TS with master...
I20260812 16:33:55.898528 1357 heartbeater.cc:511] Master 127.19.238.126:42439 requested a full tablet report, sending...
I20260812 16:33:55.899006 813 ts_manager.cc:194] Registered new tserver with Master: 5816e8856725446a9825f22b88b1d46e (127.19.238.67:33001)
I20260812 16:33:55.899545 813 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:47651
I20260812 16:33:55.901223 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:33:55.921886 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:55.912321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38290 (local address 127.19.238.126:42439)
0812 16:33:55.912392 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:55.912395 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:55.912624 (+ 229us) server_negotiation.cc:407] Connection header received
0812 16:33:55.912760 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:55.912761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:55.912775 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:55.912826 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:33:55.913775 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.914594 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.915957 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.916276 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.916358 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:55.918328 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:33:55.918331 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:33:55.918331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:33:55.918340 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:33:55.920303 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:55.920792 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:55.920794 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:55.920794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:55.920882 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:33:55.921139 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:33:55.921141 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:33:55.921141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:33:55.921294 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:33:55.921384 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:55.921667 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 16:33:55.921762 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 16:33:55.924283 813 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38290:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:33:55.926992 813 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:33:55.943545 1371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:55.932141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:36307 (local address 127.0.0.1:35604)
0812 16:33:55.934271 (+ 2130us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:55.934281 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:55.934328 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:55.934794 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:55.934797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:55.934812 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:33:55.934895 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:55.934898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.936380 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.936381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:55.937867 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:55.937870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.943104 (+ 5234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.943105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:55.943261 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:55.943281 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 16:33:55.943368 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 16:33:55.943435 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2113,"thread_start_us":97,"threads_started":1}
I20260812 16:33:55.944100 1373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:55.932435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35604 (local address 127.19.238.65:36307)
0812 16:33:55.933929 (+ 1494us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:55.933935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:55.934392 (+ 457us) server_negotiation.cc:407] Connection header received
0812 16:33:55.934437 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:55.934440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:55.934459 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:55.934536 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:33:55.935002 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.936262 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.937981 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.938859 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.938944 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:55.939016 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:55.943951 (+ 4935us) server_negotiation.cc:299] Negotiation successful
0812 16:33:55.944023 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1451,"thread_start_us":60,"threads_started":1}
I20260812 16:33:55.948587 970 tablet_service.cc:1467] Processing CreateTablet for tablet 9c45e40dfa834627a9adcbd3a65ee8f3 (DEFAULT_TABLE table=test-table [id=a96620100a584ecca6014131d2cc2de5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:33:55.948904 1370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:55.932396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:33159 (local address 127.0.0.1:58514)
0812 16:33:55.934651 (+ 2255us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:55.934662 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:55.934711 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:55.939423 (+ 4712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:55.939424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:55.939426 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:33:55.939483 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:55.939484 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.940931 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.940932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:55.942302 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:55.942306 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.948608 (+ 6302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.948609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:55.948688 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:55.948694 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 16:33:55.948755 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 16:33:55.948815 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2230,"thread_start_us":75,"threads_started":1}
I20260812 16:33:55.949054 970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c45e40dfa834627a9adcbd3a65ee8f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:33:55.950505 1372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:55.932138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:33001 (local address 127.0.0.1:37884)
0812 16:33:55.934019 (+ 1881us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:55.934072 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:55.934151 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:55.942777 (+ 8626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:55.942778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:55.942781 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:33:55.942857 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:55.942859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.945773 (+ 2914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.945774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:55.947159 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:55.947162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.950210 (+ 3048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.950211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:55.950304 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:55.950309 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:33:55.950369 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 16:33:55.950416 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1844,"thread_start_us":114,"threads_started":1}
I20260812 16:33:55.950865 1375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:55.932982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58514 (local address 127.19.238.66:33159)
0812 16:33:55.939135 (+ 6153us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:55.939141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:55.939153 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:33:55.939203 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:55.939205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:55.939224 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:55.939314 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:33:55.939572 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.940799 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.947308 (+ 6509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.948386 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.948474 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:55.948564 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:55.950701 (+ 2137us) server_negotiation.cc:299] Negotiation successful
0812 16:33:55.950755 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6102,"thread_start_us":90,"threads_started":1}
I20260812 16:33:55.951124 1374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:55.932628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37884 (local address 127.19.238.67:33001)
0812 16:33:55.942503 (+ 9875us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:55.942509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:55.942522 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:33:55.942566 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:55.942569 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:55.942588 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:55.942677 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:33:55.944415 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.945650 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.949151 (+ 3501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.949993 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.950081 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:55.950168 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:55.950984 (+ 816us) server_negotiation.cc:299] Negotiation successful
0812 16:33:55.951038 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9817,"thread_start_us":91,"threads_started":1}
I20260812 16:33:55.951644 1131 tablet_service.cc:1467] Processing CreateTablet for tablet 9c45e40dfa834627a9adcbd3a65ee8f3 (DEFAULT_TABLE table=test-table [id=a96620100a584ecca6014131d2cc2de5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:33:55.952054 1131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c45e40dfa834627a9adcbd3a65ee8f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:33:55.951774 1278 tablet_service.cc:1467] Processing CreateTablet for tablet 9c45e40dfa834627a9adcbd3a65ee8f3 (DEFAULT_TABLE table=test-table [id=a96620100a584ecca6014131d2cc2de5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:33:55.952306 1278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c45e40dfa834627a9adcbd3a65ee8f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:33:55.960065 1376 tablet_bootstrap.cc:492] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 41bd6873229a4825b3d102ac8ae020cd: Bootstrap starting.
I20260812 16:33:55.961061 1376 tablet_bootstrap.cc:654] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 41bd6873229a4825b3d102ac8ae020cd: Neither blocks nor log segments found. Creating new log.
I20260812 16:33:55.961472 1376 log.cc:824] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 41bd6873229a4825b3d102ac8ae020cd: Log is configured to *not* fsync() on all Append() calls
I20260812 16:33:55.964043 1377 tablet_bootstrap.cc:492] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e: Bootstrap starting.
I20260812 16:33:55.965193 1377 tablet_bootstrap.cc:654] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e: Neither blocks nor log segments found. Creating new log.
I20260812 16:33:55.965580 1377 log.cc:824] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:33:55.967078 1378 tablet_bootstrap.cc:492] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357: Bootstrap starting.
I20260812 16:33:55.968428 1378 tablet_bootstrap.cc:654] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357: Neither blocks nor log segments found. Creating new log.
I20260812 16:33:55.969228 1378 log.cc:824] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357: Log is configured to *not* fsync() on all Append() calls
I20260812 16:33:55.969274 1377 tablet_bootstrap.cc:492] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e: No bootstrap required, opened a new log
I20260812 16:33:55.969344 1377 ts_tablet_manager.cc:1397] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 16:33:55.970840 1376 tablet_bootstrap.cc:492] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 41bd6873229a4825b3d102ac8ae020cd: No bootstrap required, opened a new log
I20260812 16:33:55.970949 1376 ts_tablet_manager.cc:1397] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 41bd6873229a4825b3d102ac8ae020cd: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260812 16:33:55.971661 1377 raft_consensus.cc:348] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5816e8856725446a9825f22b88b1d46e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33001 } } peers { permanent_uuid: "7c3fbee1e1994b4e9917e7abd14ba357" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 33159 } } peers { permanent_uuid: "41bd6873229a4825b3d102ac8ae020cd" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 36307 } }
I20260812 16:33:55.971904 1377 raft_consensus.cc:374] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:33:55.971951 1377 raft_consensus.cc:729] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5816e8856725446a9825f22b88b1d46e, State: Initialized, Role: FOLLOWER
I20260812 16:33:55.972123 1377 consensus_queue.cc:260] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5816e8856725446a9825f22b88b1d46e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33001 } } peers { permanent_uuid: "7c3fbee1e1994b4e9917e7abd14ba357" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 33159 } } peers { permanent_uuid: "41bd6873229a4825b3d102ac8ae020cd" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 36307 } }
I20260812 16:33:55.972426 1377 ts_tablet_manager.cc:1428] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:33:55.972548 1357 heartbeater.cc:503] Master 127.19.238.126:42439 was elected leader, sending a full tablet report...
I20260812 16:33:55.974006 1376 raft_consensus.cc:348] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 41bd6873229a4825b3d102ac8ae020cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5816e8856725446a9825f22b88b1d46e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33001 } } peers { permanent_uuid: "7c3fbee1e1994b4e9917e7abd14ba357" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 33159 } } peers { permanent_uuid: "41bd6873229a4825b3d102ac8ae020cd" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 36307 } }
I20260812 16:33:55.974197 1376 raft_consensus.cc:374] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 41bd6873229a4825b3d102ac8ae020cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:33:55.974223 1376 raft_consensus.cc:729] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 41bd6873229a4825b3d102ac8ae020cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41bd6873229a4825b3d102ac8ae020cd, State: Initialized, Role: FOLLOWER
I20260812 16:33:55.974352 1376 consensus_queue.cc:260] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 41bd6873229a4825b3d102ac8ae020cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5816e8856725446a9825f22b88b1d46e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33001 } } peers { permanent_uuid: "7c3fbee1e1994b4e9917e7abd14ba357" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 33159 } } peers { permanent_uuid: "41bd6873229a4825b3d102ac8ae020cd" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 36307 } }
I20260812 16:33:55.974663 1051 heartbeater.cc:503] Master 127.19.238.126:42439 was elected leader, sending a full tablet report...
I20260812 16:33:55.974643 1376 ts_tablet_manager.cc:1428] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 41bd6873229a4825b3d102ac8ae020cd: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260812 16:33:55.976842 1378 tablet_bootstrap.cc:492] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357: No bootstrap required, opened a new log
I20260812 16:33:55.976998 1378 ts_tablet_manager.cc:1397] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260812 16:33:55.979091 1378 raft_consensus.cc:348] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5816e8856725446a9825f22b88b1d46e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33001 } } peers { permanent_uuid: "7c3fbee1e1994b4e9917e7abd14ba357" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 33159 } } peers { permanent_uuid: "41bd6873229a4825b3d102ac8ae020cd" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 36307 } }
I20260812 16:33:55.979405 1378 raft_consensus.cc:374] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:33:55.979496 1378 raft_consensus.cc:729] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c3fbee1e1994b4e9917e7abd14ba357, State: Initialized, Role: FOLLOWER
I20260812 16:33:55.979702 1378 consensus_queue.cc:260] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5816e8856725446a9825f22b88b1d46e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33001 } } peers { permanent_uuid: "7c3fbee1e1994b4e9917e7abd14ba357" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 33159 } } peers { permanent_uuid: "41bd6873229a4825b3d102ac8ae020cd" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 36307 } }
I20260812 16:33:55.980108 1378 ts_tablet_manager.cc:1428] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:33:55.980279 1206 heartbeater.cc:503] Master 127.19.238.126:42439 was elected leader, sending a full tablet report...
I20260812 16:33:55.994161 1384 raft_consensus.cc:482] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:33:55.994345 1384 raft_consensus.cc:504] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5816e8856725446a9825f22b88b1d46e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33001 } } peers { permanent_uuid: "7c3fbee1e1994b4e9917e7abd14ba357" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 33159 } } peers { permanent_uuid: "41bd6873229a4825b3d102ac8ae020cd" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 36307 } }
I20260812 16:33:55.994786 1384 leader_election.cc:288] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5816e8856725446a9825f22b88b1d46e (127.19.238.67:33001), 41bd6873229a4825b3d102ac8ae020cd (127.19.238.65:36307)
I20260812 16:33:56.000259 1210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:55.995233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:33001 (local address 127.19.238.66:55299)
0812 16:33:55.995394 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:55.995408 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:55.995464 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:55.996085 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:55.996086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:55.996089 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:33:55.996169 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:55.996171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.997460 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.997461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:55.998812 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:55.998816 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.999938 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.999939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:56.000051 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:56.000056 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:33:56.000120 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 16:33:56.000166 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137}
I20260812 16:33:56.001220 1374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:55.995558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:55299 (local address 127.19.238.67:33001)
0812 16:33:55.995609 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:55.995611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:55.995621 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:33:55.995652 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:55.995653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:55.995665 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:55.995719 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:33:55.996280 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.997113 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.999163 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:55.999770 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:55.999884 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:55.999896 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:56.001091 (+ 1195us) server_negotiation.cc:299] Negotiation successful
0812 16:33:56.001144 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 16:33:56.001552 1304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c45e40dfa834627a9adcbd3a65ee8f3" candidate_uuid: "7c3fbee1e1994b4e9917e7abd14ba357" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5816e8856725446a9825f22b88b1d46e" is_pre_election: true
I20260812 16:33:56.001729 1304 raft_consensus.cc:2454] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c3fbee1e1994b4e9917e7abd14ba357 in term 0.
I20260812 16:33:56.002224 1077 leader_election.cc:302] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5816e8856725446a9825f22b88b1d46e, 7c3fbee1e1994b4e9917e7abd14ba357; no voters:
I20260812 16:33:56.005040 1384 raft_consensus.cc:2781] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:33:56.005127 1384 raft_consensus.cc:482] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:33:56.005155 1384 raft_consensus.cc:3037] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:33:56.006352 1373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:55.995996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:53121 (local address 127.19.238.65:36307)
0812 16:33:55.996043 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:55.996045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:56.000488 (+ 4443us) server_negotiation.cc:407] Connection header received
0812 16:33:56.000524 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:56.000526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:56.000537 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:56.000589 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:33:56.002697 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:56.003500 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:56.005004 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:56.005623 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:56.005714 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:56.005726 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:56.006225 (+ 499us) server_negotiation.cc:299] Negotiation successful
0812 16:33:56.006283 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 16:33:56.006026 1210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:55.997265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:36307 (local address 127.19.238.66:53121)
0812 16:33:56.000359 (+ 3094us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:56.000372 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:33:56.000422 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:33:56.002546 (+ 2124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:33:56.002547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:33:56.002550 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:33:56.002588 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:56.002590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:56.003940 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:56.003941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:56.004898 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:33:56.004901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:33:56.005761 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:33:56.005762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:33:56.005841 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:56.005845 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:33:56.005901 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 16:33:56.005942 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3081,"thread_start_us":139,"threads_started":1}
I20260812 16:33:56.006887 994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c45e40dfa834627a9adcbd3a65ee8f3" candidate_uuid: "7c3fbee1e1994b4e9917e7abd14ba357" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41bd6873229a4825b3d102ac8ae020cd" is_pre_election: true
I20260812 16:33:56.007115 994 raft_consensus.cc:2454] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 41bd6873229a4825b3d102ac8ae020cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c3fbee1e1994b4e9917e7abd14ba357 in term 0.
I20260812 16:33:56.006233 1384 raft_consensus.cc:504] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5816e8856725446a9825f22b88b1d46e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33001 } } peers { permanent_uuid: "7c3fbee1e1994b4e9917e7abd14ba357" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 33159 } } peers { permanent_uuid: "41bd6873229a4825b3d102ac8ae020cd" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 36307 } }
I20260812 16:33:56.007961 1304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c45e40dfa834627a9adcbd3a65ee8f3" candidate_uuid: "7c3fbee1e1994b4e9917e7abd14ba357" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5816e8856725446a9825f22b88b1d46e"
I20260812 16:33:56.008059 1304 raft_consensus.cc:3037] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:33:56.009109 1304 raft_consensus.cc:2454] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c3fbee1e1994b4e9917e7abd14ba357 in term 1.
I20260812 16:33:56.009415 1077 leader_election.cc:302] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5816e8856725446a9825f22b88b1d46e, 7c3fbee1e1994b4e9917e7abd14ba357; no voters:
I20260812 16:33:56.009909 994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c45e40dfa834627a9adcbd3a65ee8f3" candidate_uuid: "7c3fbee1e1994b4e9917e7abd14ba357" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41bd6873229a4825b3d102ac8ae020cd"
I20260812 16:33:56.010020 994 raft_consensus.cc:3037] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 41bd6873229a4825b3d102ac8ae020cd [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:33:56.011044 994 raft_consensus.cc:2454] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 41bd6873229a4825b3d102ac8ae020cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c3fbee1e1994b4e9917e7abd14ba357 in term 1.
I20260812 16:33:56.011330 1384 leader_election.cc:288] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [CANDIDATE]: Term 1 election: Requested vote from peers 5816e8856725446a9825f22b88b1d46e (127.19.238.67:33001), 41bd6873229a4825b3d102ac8ae020cd (127.19.238.65:36307)
I20260812 16:33:56.011456 1384 raft_consensus.cc:2781] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:33:56.011544 1384 raft_consensus.cc:686] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [term 1 LEADER]: Becoming Leader. State: Replica: 7c3fbee1e1994b4e9917e7abd14ba357, State: Running, Role: LEADER
I20260812 16:33:56.011708 1384 consensus_queue.cc:237] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5816e8856725446a9825f22b88b1d46e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33001 } } peers { permanent_uuid: "7c3fbee1e1994b4e9917e7abd14ba357" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 33159 } } peers { permanent_uuid: "41bd6873229a4825b3d102ac8ae020cd" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 36307 } }
I20260812 16:33:56.012920 813 catalog_manager.cc:5495] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c3fbee1e1994b4e9917e7abd14ba357 (127.19.238.66). New cstate: current_term: 1 leader_uuid: "7c3fbee1e1994b4e9917e7abd14ba357" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5816e8856725446a9825f22b88b1d46e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c3fbee1e1994b4e9917e7abd14ba357" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 33159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41bd6873229a4825b3d102ac8ae020cd" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 36307 } health_report { overall_health: UNKNOWN } } }
I20260812 16:33:56.031428 1375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:56.027751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58526 (local address 127.19.238.66:33159)
0812 16:33:56.027859 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 16:33:56.027869 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:33:56.027880 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:33:56.027929 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:33:56.027930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:33:56.027959 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:33:56.028069 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:33:56.028295 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:56.029047 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:56.030222 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:33:56.030543 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:33:56.030611 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:33:56.030826 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:33:56.030969 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:33:56.031069 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 16:33:56.031266 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 16:33:56.031289 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 16:33:56.034709 994 raft_consensus.cc:1261] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 41bd6873229a4825b3d102ac8ae020cd [term 1 FOLLOWER]: Refusing update from remote peer 7c3fbee1e1994b4e9917e7abd14ba357: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:33:56.035230 1386 consensus_queue.cc:1035] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41bd6873229a4825b3d102ac8ae020cd" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 36307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:33:56.038070 1304 raft_consensus.cc:1261] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e [term 1 FOLLOWER]: Refusing update from remote peer 7c3fbee1e1994b4e9917e7abd14ba357: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:33:56.038523 1386 consensus_queue.cc:1035] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5816e8856725446a9825f22b88b1d46e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:33:56.041360 1389 mvcc.cc:205] Tried to move back new op lower bound from 7317718777994444800 to 7317718777905274880. Current Snapshot: MvccSnapshot[applied={T|T < 7317718777994444800}]
I20260812 16:33:56.041533 1390 mvcc.cc:205] Tried to move back new op lower bound from 7317718777994444800 to 7317718777905274880. Current Snapshot: MvccSnapshot[applied={T|T < 7317718777994444800}]
I20260812 16:33:56.044727 813 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:33:56.044838 813 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:33:56.045773 813 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0ca3b42d7b284b828fd7a755652ad3d8: Sending DeleteTablet for 3 replicas of tablet 9c45e40dfa834627a9adcbd3a65ee8f3
I20260812 16:33:56.046346 1278 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c45e40dfa834627a9adcbd3a65ee8f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:33:56 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:37884
I20260812 16:33:56.046439 1131 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c45e40dfa834627a9adcbd3a65ee8f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:33:56 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:58514
I20260812 16:33:56.046674 970 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c45e40dfa834627a9adcbd3a65ee8f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:33:56 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:35604
I20260812 16:33:56.047523 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 898
I20260812 16:33:56.054085 1399 tablet_replica.cc:330] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357: stopping tablet replica
I20260812 16:33:56.054224 1399 raft_consensus.cc:2229] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:33:56.054359 1399 raft_consensus.cc:2258] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:33:56.055745 1392 mvcc.cc:205] Tried to move back new op lower bound from 7317718777994444800 to 7317718777905274880. Current Snapshot: MvccSnapshot[applied={T|T < 7317718777994444800}]
I20260812 16:33:56.059284 1399 ts_tablet_manager.cc:1905] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:33:56.064141 1398 tablet_replica.cc:330] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e: stopping tablet replica
I20260812 16:33:56.064289 1398 raft_consensus.cc:2229] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:33:56.064378 1398 raft_consensus.cc:2258] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 16:33:56.064767 798 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.238.65:36307 (error 108)
I20260812 16:33:56.065183 1399 ts_tablet_manager.cc:1918] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:33:56.065378 1399 log.cc:1196] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/9c45e40dfa834627a9adcbd3a65ee8f3
I20260812 16:33:56.065793 1399 ts_tablet_manager.cc:1939] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 7c3fbee1e1994b4e9917e7abd14ba357: Deleting consensus metadata
I20260812 16:33:56.066382 795 catalog_manager.cc:4841] TS 7c3fbee1e1994b4e9917e7abd14ba357 (127.19.238.66:33159): tablet 9c45e40dfa834627a9adcbd3a65ee8f3 (table test-table [id=a96620100a584ecca6014131d2cc2de5]) successfully deleted
I20260812 16:33:56.068255 1372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:33:56.068119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:36307 (local address 127.0.0.1:35608)
0812 16:33:56.068176 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 16:33:56.068217 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:36307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 16:33:56.068506 798 catalog_manager.cc:4584] TS 41bd6873229a4825b3d102ac8ae020cd (127.19.238.65:36307): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9c45e40dfa834627a9adcbd3a65ee8f3: Network error: Client connection negotiation failed: client connection to 127.19.238.65:36307: connect: Connection refused (error 111)
I20260812 16:33:56.072184 1398 ts_tablet_manager.cc:1905] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:33:56.073954 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 1057
I20260812 16:33:56.076992 1398 ts_tablet_manager.cc:1918] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:33:56.077093 1398 log.cc:1196] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/9c45e40dfa834627a9adcbd3a65ee8f3
I20260812 16:33:56.077800 1398 ts_tablet_manager.cc:1939] T 9c45e40dfa834627a9adcbd3a65ee8f3 P 5816e8856725446a9825f22b88b1d46e: Deleting consensus metadata
I20260812 16:33:56.078370 795 catalog_manager.cc:4841] TS 5816e8856725446a9825f22b88b1d46e (127.19.238.67:33001): tablet 9c45e40dfa834627a9adcbd3a65ee8f3 (table test-table [id=a96620100a584ecca6014131d2cc2de5]) successfully deleted
I20260812 16:33:56.098966 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 1212
I20260812 16:33:56.106307 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 774
I20260812 16:33:56.114291 20409 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 16:33:56.458941 20409 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 16:33:56.459057 20409 mini_ranger.cc:66] Stopping Ranger...
I20260812 16:33:56.804620 20409 mini_ranger.cc:68] Stopped Ranger
2026-08-12 16:33:56.805 UTC [32764] LOG: received smart shutdown request
2026-08-12 16:33:56.809 UTC [32764] LOG: background worker "logical replication launcher" (PID 306) exited with exit code 1
2026-08-12 16:33:56.809 UTC [300] LOG: shutting down
2026-08-12 16:33:56.991 UTC [32764] LOG: database system is shut down
2026-08-12T16:33:56Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50327 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:33:57 dist-test-slave-mvvj krb5kdc[1414](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:33:57 dist-test-slave-mvvj krb5kdc[1414](info): set up 2 sockets
Aug 12 16:33:57 dist-test-slave-mvvj krb5kdc[1414](info): commencing operation
krb5kdc: starting...
W20260812 16:33:58.578402 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.446s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:33:58 dist-test-slave-mvvj krb5kdc[1414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:33:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:33:58Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.19.238.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.19.238.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.19.238.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/rangeradmin_127.19.238.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.19.238.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/rangeradmin_127.19.238.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.19.238.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.19.238.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.19.238.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/rangerlookup_127.19.238.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.19.238.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/rangerlookup_127.19.238.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.238.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.19.238.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.238.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/HTTP_127.19.238.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.238.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/HTTP_127.19.238.84@KRBTEST.COM.keytab.
I20260812 16:33:58.770957 20409 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskt79009/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/postgres -l logfile start
2026-08-12 16:34:01.505 UTC [1450] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 16:34:01.505 UTC [1450] LOG: listening on IPv4 address "127.19.238.84", port 56229
2026-08-12 16:34:01.509 UTC [1450] LOG: listening on Unix socket "/tmp/.s.PGSQL.56229"
2026-08-12 16:34:01.530 UTC [1453] LOG: database system was shut down at 2026-08-12 16:33:59 UTC
2026-08-12 16:34:01.534 UTC [1450] LOG: database system is ready to accept connections
I20260812 16:34:02.598188 20409 mini_postgres.cc:96] Postgres bound to 56229
2026-08-12 16:34:02.605 UTC [1461] FATAL: database "slave" does not exist
127.19.238.84:56229 - accepting connections
I20260812 16:34:02.606169 20409 mini_ranger.cc:161] Starting Ranger...
I20260812 16:34:02.622486 20409 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 16:34:03.173116 20409 mini_ranger.cc:87] Created ranger Postgres database
I20260812 16:34:03.173229 20409 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ranger-admin
2026-08-12 16:34:03,466 [I] DB FLAVOR :POSTGRES
2026-08-12 16:34:03,467 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 16:34:03,467 [I] Checking connection..
2026-08-12 16:34:03,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 16:34:03,750 [I] Checking connection passed.
2026-08-12 16:34:03,751 [I] --------- Verifying version history table ---------
2026-08-12 16:34:03,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 16:34:04,027 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 16:34:04,027 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 16:34:04,274 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 16:34:04,274 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 16:34:04,275 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 16:34:04.543 UTC [1532] WARNING: there is no transaction in progress
2026-08-12 16:34:04,551 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 16:34:04,551 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 16:34:04,830 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 16:34:04,831 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 16:34:04,831 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 16:34:05,107 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 16:34:05,351 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-mvvj.c.gcp-upstream.internal','N') ;"
2026-08-12 16:34:05,603 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 16:34:05,859 [I] Table x_portal_user does not exist in database ranger
2026-08-12 16:34:05,860 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 16:34:06,135 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 16:34:06,136 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 16:34:06,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 16:34:06,632 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 16:34:06,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 16:34:07.727 UTC [1654] WARNING: there is no transaction in progress
2026-08-12 16:34:07.745 UTC [1654] WARNING: there is no transaction in progress
2026-08-12 16:34:07.765 UTC [1654] WARNING: there is no transaction in progress
2026-08-12 16:34:07.782 UTC [1654] WARNING: there is no transaction in progress
2026-08-12 16:34:07.800 UTC [1654] WARNING: there is no transaction in progress
2026-08-12 16:34:07.821 UTC [1654] WARNING: there is no transaction in progress
2026-08-12 16:34:07.884 UTC [1654] WARNING: there is no transaction in progress
2026-08-12 16:34:07.893 UTC [1654] WARNING: there is no transaction in progress
2026-08-12 16:34:07.909 UTC [1654] WARNING: there is no transaction in progress
2026-08-12 16:34:07.920 UTC [1654] WARNING: there is no transaction in progress
2026-08-12 16:34:07.931 UTC [1654] WARNING: there is no transaction in progress
2026-08-12 16:34:07.947 UTC [1654] WARNING: there is no transaction in progress
2026-08-12 16:34:07.958 UTC [1654] WARNING: there is no transaction in progress
2026-08-12 16:34:07.965 UTC [1654] WARNING: there is no transaction in progress
2026-08-12 16:34:07.974 UTC [1654] WARNING: there is no transaction in progress
2026-08-12 16:34:07.982 UTC [1654] WARNING: there is no transaction in progress
2026-08-12 16:34:08,778 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 16:34:08,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 16:34:09,042 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 16:34:09,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 16:34:09,303 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 16:34:09,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 16:34:09,574 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 16:34:09,575 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 16:34:09,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 16:34:10,104 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-mvvj.c.gcp-upstream.internal';"
2026-08-12 16:34:10,360 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 16:34:10,361 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 16:34:10,609 [I] DB_PATCHES have already been applied
I20260812 16:34:10.615314 20409 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ranger-admin:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskt79009/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 16:34:10.615386 20409 mini_ranger.cc:193] Using host: 127.19.238.84
I20260812 16:34:10.619210 20409 mini_ranger.cc:227] Ranger admin URL: http://127.19.238.84:52487
Aug 12, 2026 4:34:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 4:34:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 4:34:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 4:34:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 4:34:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 4:34:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 4:34:11 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 16:34:11.838392 20409 mini_ranger.cc:160] Time spent starting Ranger: real 9.232s user 0.000s sys 0.011s
Aug 12, 2026 4:34:31 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 4:34:31 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 4:34:31 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 4:34:31 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 4:34:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 4:34:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 4:34:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 4:34:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 4:34:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 4:34:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 4:34:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 4:34:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 4:34:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 4:34:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 4:34:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 4:34:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 4:34:31 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 4:34:32 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 16:34:33.604202 20409 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.19.238.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.19.238.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.19.238.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/rangerkms_127.19.238.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.19.238.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/rangerkms_127.19.238.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.238.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.19.238.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.238.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/HTTP_127.19.238.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.238.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/HTTP_127.19.238.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 16:34:33 dist-test-slave-mvvj krb5kdc[1414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552473, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 16:34:33.725195 20409 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 16:34:33.738426 20409 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 16:34:34.764850 20409 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 16:34:34.764950 20409 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ranger-kms
I20260812 16:34:34.764972 20409 mini_ranger_kms.cc:227] Using postgres at 127.19.238.84:56229
2026-08-12 16:34:34,941 [I] DB FLAVOR :POSTGRES
2026-08-12 16:34:34,942 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 16:34:34,942 [I] Checking connection
2026-08-12 16:34:34,942 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 16:34:35,222 [I] connection success
2026-08-12 16:34:35,223 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 16:34:35,223 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 16:34:35,223 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 16:34:35,469 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 16:34:35,469 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 16:34:35,469 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 16:34:35,469 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.238.84:56229/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 16:34:35,743 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 16:34:36.264505 20409 mini_ranger_kms.cc:310] Created kms service
I20260812 16:34:36.479563 20409 mini_ranger_kms.cc:326] Created kudu user
I20260812 16:34:36.525893 20409 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 16:34:36 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552473, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
I20260812 16:34:36.966442 20409 mini_ranger_kms.cc:384] Added ranger policy
I20260812 16:34:36.966663 20409 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ranger-kms:/tmp/dist-test-taskt79009/build/release/bin/postgresql.jar:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskt79009/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 16:34:36.966686 20409 mini_ranger_kms.cc:240] Using host: 127.19.238.84
I20260812 16:34:36.971117 20409 mini_ranger_kms.cc:276] Ranger KMS PID: 1852
I20260812 16:34:36.971220 20409 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.19.238.84:56503
Aug 12, 2026 4:34:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 4:34:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 4:34:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 4:34:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 4:34:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 4:34:38 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260812 16:34:39.395355 20409 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.670s user 0.000s sys 0.160s
I20260812 16:34:39.395473 20409 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 16:34:39.395536 20409 mini_ranger_kms.cc:401] 127.19.238.84:56503/kms/v1/keys
Aug 12 16:34:39 dist-test-slave-mvvj krb5kdc[1414](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786552479, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.238.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 16:34:39 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552479, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.238.84@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
Aug 12 16:34:40 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552479, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.238.84@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
Aug 12 16:34:42 dist-test-slave-mvvj krb5kdc[1414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:34:42.679666 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:45001
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:45001
--ranger_config_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.238.84:56503/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:34:42.774603 1911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:42.774829 1911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:42.774852 1911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:42.776098 1911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:34:42.776139 1911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:42.776154 1911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:42.776166 1911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:34:42.776180 1911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:34:42.777837 1911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/wal
--ranger_kms_url=127.19.238.84:56503/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:45001
--ranger_config_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:45001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:42.778115 1911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:42.778391 1911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 16:34:42.783855 1911 server_base.cc:1034] running on GCE node
W20260812 16:34:42.783901 1919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:42.783803 1917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:42.783779 1916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:42.784312 1911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:42.784641 1911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:42.785852 1911 hybrid_clock.cc:648] HybridClock initialized: now 1786552482785821 us; error 32 us; skew 500 ppm
Aug 12 16:34:42 dist-test-slave-mvvj krb5kdc[1414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552482, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:42.793565 1911 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:34:42.794137 1911 webserver.cc:460] Webserver started at http://127.19.238.126:39103/ using document root <none> and password file <none>
I20260812 16:34:42.794363 1911 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:42.794407 1911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:42.794512 1911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:34:42 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552482, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
I20260812 16:34:42.862908 1911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "7640a127d2f44227b38cd312c486a133"
format_stamp: "Formatted at 2026-08-12 16:34:42 on dist-test-slave-mvvj"
server_key: "9d8f94957faa472f066746339ee973bc"
server_key_iv: "21d57d8f0a1ec75f5471de207eef0c95"
server_key_version: "kuduclusterkey@0"
I20260812 16:34:42.863351 1911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "7640a127d2f44227b38cd312c486a133"
format_stamp: "Formatted at 2026-08-12 16:34:42 on dist-test-slave-mvvj"
server_key: "9d8f94957faa472f066746339ee973bc"
server_key_iv: "21d57d8f0a1ec75f5471de207eef0c95"
server_key_version: "kuduclusterkey@0"
I20260812 16:34:42.864959 1911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260812 16:34:42.881035 1926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:42.881354 1911 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:34:42.881494 1911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "7640a127d2f44227b38cd312c486a133"
format_stamp: "Formatted at 2026-08-12 16:34:42 on dist-test-slave-mvvj"
server_key: "9d8f94957faa472f066746339ee973bc"
server_key_iv: "21d57d8f0a1ec75f5471de207eef0c95"
server_key_version: "kuduclusterkey@0"
I20260812 16:34:42.881613 1911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:42.898427 1911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:42.900811 1911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:42.901270 1911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:42.906380 1911 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:45001
I20260812 16:34:42.906418 1982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:45001 every 8 connection(s)
I20260812 16:34:42.906855 1911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:34:42.907653 1983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:34:42.908531 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 1911
I20260812 16:34:42.908638 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:34:42.908835 20409 external_mini_cluster.cc:1428] Setting key b7a5bebf55806d052c4d6c19b4c35996
I20260812 16:34:42.910650 1983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133: Bootstrap starting.
I20260812 16:34:42.911403 1983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133: Neither blocks nor log segments found. Creating new log.
I20260812 16:34:42.911736 1983 log.cc:824] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133: Log is configured to *not* fsync() on all Append() calls
I20260812 16:34:42.912712 1983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133: No bootstrap required, opened a new log
I20260812 16:34:42.914820 1983 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7640a127d2f44227b38cd312c486a133" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45001 } }
I20260812 16:34:42.915024 1983 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:34:42.915060 1983 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7640a127d2f44227b38cd312c486a133, State: Initialized, Role: FOLLOWER
I20260812 16:34:42.915256 1983 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7640a127d2f44227b38cd312c486a133" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45001 } }
I20260812 16:34:42.915349 1983 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:34:42.915387 1983 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:34:42.915459 1983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:34:42.916282 1983 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7640a127d2f44227b38cd312c486a133" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45001 } }
I20260812 16:34:42.916409 1983 leader_election.cc:302] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7640a127d2f44227b38cd312c486a133; no voters:
I20260812 16:34:42.916602 1983 leader_election.cc:288] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:34:42.916658 1988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:34:42.916949 1988 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [term 1 LEADER]: Becoming Leader. State: Replica: 7640a127d2f44227b38cd312c486a133, State: Running, Role: LEADER
I20260812 16:34:42.917027 1983 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:34:42.917095 1988 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7640a127d2f44227b38cd312c486a133" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45001 } }
I20260812 16:34:42.917500 1988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7640a127d2f44227b38cd312c486a133" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7640a127d2f44227b38cd312c486a133" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45001 } } }
I20260812 16:34:42.917640 1988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [sys.catalog]: This master's current role is: LEADER
I20260812 16:34:42.917819 1988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7640a127d2f44227b38cd312c486a133. Latest consensus state: current_term: 1 leader_uuid: "7640a127d2f44227b38cd312c486a133" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7640a127d2f44227b38cd312c486a133" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45001 } } }
I20260812 16:34:42.917935 1988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [sys.catalog]: This master's current role is: LEADER
I20260812 16:34:42.918099 1996 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 16:34:42 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:34:42.918645 1996 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:34:42.918982 1983 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 16:34:42.927824 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:42.909419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56668 (local address 127.19.238.126:45001)
0812 16:34:42.909621 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:42.909631 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:42.910986 (+ 1355us) server_negotiation.cc:407] Connection header received
0812 16:34:42.911574 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:42.911584 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:42.911650 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:42.911811 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:42.912611 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:42.914076 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:42.915175 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:42.915801 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:42.915938 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:42.919269 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:42.919274 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:42.919275 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:42.919290 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:42.925300 (+ 6010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:42.925859 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:42.925862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:42.925863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:42.926021 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:42.926264 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:42.926265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:42.926265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:42.926688 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:42.927337 (+ 649us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:42.927423 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 16:34:42.927583 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":82,"threads_started":1}
I20260812 16:34:42.924525 2004 mvcc.cc:205] Tried to move back new op lower bound from 7317718970054963200 to 7317718970029035520. Current Snapshot: MvccSnapshot[applied={T|T < 7317718970054963200}]
I20260812 16:34:42.932346 1996 catalog_manager.cc:1347] Generated new cluster ID: f107178afb104e45ba3ace6ab7265f14
I20260812 16:34:42.932408 1996 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:34:42.947053 1996 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:34:42.948288 1996 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:34:42.958406 1996 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133: Generated new TSK 0
I20260812 16:34:42.958738 1996 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
I20260812 16:34:43.704221 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.238.84:56503/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:45001
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:34:43.826313 2031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:43.826581 2031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:43.826627 2031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:43.828634 2031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:43.828732 2031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:43.828787 2031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:34:43.830966 2031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.19.238.84:56503/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:45001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:43.831243 2031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:43.831598 2031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:43.836997 2040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:43.837252 2031 server_base.cc:1034] running on GCE node
W20260812 16:34:43.837013 2036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:43.837040 2037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:43.837618 2031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:43.837946 2031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:43.839131 2031 hybrid_clock.cc:648] HybridClock initialized: now 1786552483839107 us; error 37 us; skew 500 ppm
Aug 12 16:34:43 dist-test-slave-mvvj krb5kdc[1414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552483, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:43.842109 2031 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:34:43.842617 2031 webserver.cc:460] Webserver started at http://127.19.238.65:45731/ using document root <none> and password file <none>
I20260812 16:34:43.842885 2031 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:43.842964 2031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:43.843113 2031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:34:43 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552483, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
I20260812 16:34:43.860229 2031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "5db58a364a034627a46072903439e7cb"
format_stamp: "Formatted at 2026-08-12 16:34:43 on dist-test-slave-mvvj"
server_key: "fa8af2e9a2d16d367b9a27521be92a20"
server_key_iv: "5960d95eadfe71de1600b15d93ecc5b9"
server_key_version: "kuduclusterkey@0"
I20260812 16:34:43.860918 2031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "5db58a364a034627a46072903439e7cb"
format_stamp: "Formatted at 2026-08-12 16:34:43 on dist-test-slave-mvvj"
server_key: "fa8af2e9a2d16d367b9a27521be92a20"
server_key_iv: "5960d95eadfe71de1600b15d93ecc5b9"
server_key_version: "kuduclusterkey@0"
I20260812 16:34:43.862854 2031 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 16:34:43.884033 2052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:43.884399 2031 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260812 16:34:43.884543 2031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "5db58a364a034627a46072903439e7cb"
format_stamp: "Formatted at 2026-08-12 16:34:43 on dist-test-slave-mvvj"
server_key: "fa8af2e9a2d16d367b9a27521be92a20"
server_key_iv: "5960d95eadfe71de1600b15d93ecc5b9"
server_key_version: "kuduclusterkey@0"
I20260812 16:34:43.884649 2031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:43.899132 2031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:43.901791 2031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:43.901962 2031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:43.902210 2031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:34:43.902645 2031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:34:43.902753 2031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:43.902861 2031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:34:43.902940 2031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:43.912583 2031 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:39681
I20260812 16:34:43.912667 2174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:39681 every 8 connection(s)
I20260812 16:34:43.913246 2031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
Aug 12 16:34:43 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552483, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:34:43.923130 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 2031
I20260812 16:34:43.923259 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:34:43.923563 20409 external_mini_cluster.cc:1428] Setting key d0a0d8c388fb471c51b00d7831c3000a
I20260812 16:34:43.925155 2177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:43.914134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:45001 (local address 127.19.238.65:59879)
0812 16:34:43.914347 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:43.914398 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:43.914937 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:43.915241 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:43.915245 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:43.915443 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:43.915693 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:43.915699 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:43.916854 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:43.916855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:43.917802 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:43.917805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:43.918556 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:43.918557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:43.918666 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:43.919385 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:43.919441 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:43.921056 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:43.923139 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:43.923149 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:43.923152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:43.923539 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:43.923815 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:43.923816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:43.923817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:43.923992 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:43.924322 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:43.924331 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:43.924541 (+ 210us) client_negotiation.cc:769] Sending connection context
0812 16:34:43.924666 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 16:34:43.924850 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":96,"threads_started":1}
I20260812 16:34:43.925581 2178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:43.914161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:59879 (local address 127.19.238.126:45001)
0812 16:34:43.914565 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:43.914567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:43.914582 (+ 15us) server_negotiation.cc:407] Connection header received
0812 16:34:43.915062 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:43.915063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:43.915078 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:43.915168 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:43.915867 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:43.916680 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:43.917958 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:43.918408 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:43.918498 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:43.921187 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:43.921190 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:43.921190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:43.921200 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:43.923025 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:43.923639 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:43.923641 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:43.923641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:43.923721 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:43.924075 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:43.924076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:43.924076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:43.924258 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:43.924322 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:43.925320 (+ 998us) server_negotiation.cc:299] Negotiation successful
0812 16:34:43.925435 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":93,"threads_started":1}
I20260812 16:34:43.926069 2175 heartbeater.cc:347] Connected to a master server at 127.19.238.126:45001
I20260812 16:34:43.926170 2175 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:43.926414 2175 heartbeater.cc:511] Master 127.19.238.126:45001 requested a full tablet report, sending...
I20260812 16:34:43.927028 1944 ts_manager.cc:194] Registered new tserver with Master: 5db58a364a034627a46072903439e7cb (127.19.238.65:39681)
I20260812 16:34:43.928006 1944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:59879
WARNING: no policy specified for kudu/127.19.238.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
I20260812 16:34:43.963670 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.238.84:56503/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:45001
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:34:44 dist-test-slave-mvvj krb5kdc[1414](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786552484, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 16:34:44.091323 2181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:44.091634 2181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:44.091670 2181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:44.093541 2181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:44.093649 2181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:44.093711 2181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:34:44.096069 2181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.19.238.84:56503/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:45001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:44.096484 2181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:44.096807 2181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:44.103111 2189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:44.103482 2192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:44.103986 2190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:44.104337 2181 server_base.cc:1034] running on GCE node
I20260812 16:34:44.104563 2181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:44.104864 2181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:44.106033 2181 hybrid_clock.cc:648] HybridClock initialized: now 1786552484106011 us; error 38 us; skew 500 ppm
Aug 12 16:34:44 dist-test-slave-mvvj krb5kdc[1414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552484, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:44.108826 2181 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:34:44.109387 2181 webserver.cc:460] Webserver started at http://127.19.238.66:42083/ using document root <none> and password file <none>
I20260812 16:34:44.109663 2181 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:44.109735 2181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:44.109902 2181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:34:44 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552484, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
I20260812 16:34:44.124079 2181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "3e9381d6c94f42d38b0c9d4d5ef058a3"
format_stamp: "Formatted at 2026-08-12 16:34:44 on dist-test-slave-mvvj"
server_key: "27c3895b8a12edf04e0fd5462b50d9a9"
server_key_iv: "91f29dd67af69ac3ffc1b78e16f8a94f"
server_key_version: "kuduclusterkey@0"
I20260812 16:34:44.124583 2181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "3e9381d6c94f42d38b0c9d4d5ef058a3"
format_stamp: "Formatted at 2026-08-12 16:34:44 on dist-test-slave-mvvj"
server_key: "27c3895b8a12edf04e0fd5462b50d9a9"
server_key_iv: "91f29dd67af69ac3ffc1b78e16f8a94f"
server_key_version: "kuduclusterkey@0"
I20260812 16:34:44.126180 2181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:34:44.152406 2201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:44.152690 2181 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260812 16:34:44.152797 2181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "3e9381d6c94f42d38b0c9d4d5ef058a3"
format_stamp: "Formatted at 2026-08-12 16:34:44 on dist-test-slave-mvvj"
server_key: "27c3895b8a12edf04e0fd5462b50d9a9"
server_key_iv: "91f29dd67af69ac3ffc1b78e16f8a94f"
server_key_version: "kuduclusterkey@0"
I20260812 16:34:44.152927 2181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:44.192399 2181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:44.195046 2181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:44.195267 2181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:44.195578 2181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:34:44.196024 2181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:34:44.196107 2181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:44.196170 2181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:34:44.196218 2181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:44.202898 2181 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:43033
I20260812 16:34:44.203404 2181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:34:44.206866 2323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:43033 every 8 connection(s)
I20260812 16:34:44.210934 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 2181
I20260812 16:34:44.211076 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:34:44.211308 20409 external_mini_cluster.cc:1428] Setting key 0de9a371a038c7da6425ff6c017af383
Aug 12 16:34:44 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552484, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:34:44.220934 2178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:44.207687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:50217 (local address 127.19.238.126:45001)
0812 16:34:44.207755 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:44.207757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:44.208065 (+ 308us) server_negotiation.cc:407] Connection header received
0812 16:34:44.208885 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:44.208887 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:44.208909 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:44.209004 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:44.210218 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:44.210876 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:44.212701 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:44.213061 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:44.213110 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:44.216292 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:44.216295 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:44.216295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:44.216305 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:44.218572 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:44.219348 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:44.219350 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:44.219350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:44.219437 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:44.219821 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:44.219822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:44.219822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:44.220046 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:44.220119 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:44.220624 (+ 505us) server_negotiation.cc:299] Negotiation successful
0812 16:34:44.220764 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 16:34:44.221609 2326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:44.207701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:45001 (local address 127.19.238.66:50217)
0812 16:34:44.207914 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:44.207965 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:44.208697 (+ 732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:44.209125 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:44.209130 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:44.209813 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:44.210061 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:44.210066 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:44.211017 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:44.211019 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:44.212583 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:44.212587 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:44.213233 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:44.213235 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:44.213422 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:44.214094 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:44.214107 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:44.216157 (+ 2050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:44.218803 (+ 2646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:44.218814 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:44.218817 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:44.219218 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:44.219533 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:44.219534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:44.219535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:44.219723 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:44.220166 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:44.220175 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:44.220402 (+ 227us) client_negotiation.cc:769] Sending connection context
0812 16:34:44.221068 (+ 666us) client_negotiation.cc:240] Negotiation successful
0812 16:34:44.221276 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":123,"threads_started":1}
I20260812 16:34:44.222385 2324 heartbeater.cc:347] Connected to a master server at 127.19.238.126:45001
I20260812 16:34:44.222524 2324 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:44.222740 2324 heartbeater.cc:511] Master 127.19.238.126:45001 requested a full tablet report, sending...
I20260812 16:34:44.223186 1944 ts_manager.cc:194] Registered new tserver with Master: 3e9381d6c94f42d38b0c9d4d5ef058a3 (127.19.238.66:43033)
I20260812 16:34:44.223807 1944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:50217
WARNING: no policy specified for kudu/127.19.238.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
I20260812 16:34:44.247582 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.238.84:56503/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:45001
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:34:44.359521 2329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:44.359931 2329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:44.360026 2329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:44.361856 2329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:44.361981 2329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:44.362082 2329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:34:44.364392 2329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.19.238.84:56503/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:45001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:44.364882 2329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:44.365216 2329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:44.371917 2339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:44.372236 2335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:44.372323 2336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:44.373381 2329 server_base.cc:1034] running on GCE node
I20260812 16:34:44.373644 2329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:44.373993 2329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:44.375192 2329 hybrid_clock.cc:648] HybridClock initialized: now 1786552484375167 us; error 38 us; skew 500 ppm
Aug 12 16:34:44 dist-test-slave-mvvj krb5kdc[1414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552484, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:44.378931 2329 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:34:44.379537 2329 webserver.cc:460] Webserver started at http://127.19.238.67:43443/ using document root <none> and password file <none>
I20260812 16:34:44.379899 2329 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:44.379972 2329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:44.380169 2329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:34:44 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552484, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
I20260812 16:34:44.399953 2329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "c1126bc5d2e84fc1be4b8012ae372a79"
format_stamp: "Formatted at 2026-08-12 16:34:44 on dist-test-slave-mvvj"
server_key: "e99a84f60acc8bfca6f98a371c7d6a68"
server_key_iv: "170cf4ed61b9622e2910b589b0858869"
server_key_version: "kuduclusterkey@0"
I20260812 16:34:44.400486 2329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "c1126bc5d2e84fc1be4b8012ae372a79"
format_stamp: "Formatted at 2026-08-12 16:34:44 on dist-test-slave-mvvj"
server_key: "e99a84f60acc8bfca6f98a371c7d6a68"
server_key_iv: "170cf4ed61b9622e2910b589b0858869"
server_key_version: "kuduclusterkey@0"
I20260812 16:34:44.404275 2329 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.002s
I20260812 16:34:44.419260 2347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:44.419569 2329 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 16:34:44.419678 2329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "c1126bc5d2e84fc1be4b8012ae372a79"
format_stamp: "Formatted at 2026-08-12 16:34:44 on dist-test-slave-mvvj"
server_key: "e99a84f60acc8bfca6f98a371c7d6a68"
server_key_iv: "170cf4ed61b9622e2910b589b0858869"
server_key_version: "kuduclusterkey@0"
I20260812 16:34:44.419785 2329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:44.451071 2329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:44.453630 2329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:44.453815 2329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:44.454080 2329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:34:44.454507 2329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:34:44.454566 2329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:44.454631 2329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:34:44.454661 2329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:44.462920 2329 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:45927
I20260812 16:34:44.463510 2329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:34:44.464675 2469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:45927 every 8 connection(s)
I20260812 16:34:44.467195 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 2329
I20260812 16:34:44.467304 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:34:44.467496 20409 external_mini_cluster.cc:1428] Setting key c3b0aedc20e6a1d68cd3a01d36574042
Aug 12 16:34:44 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552484, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:34:44.474655 2178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:44.465513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:33349 (local address 127.19.238.126:45001)
0812 16:34:44.465579 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:44.465581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:44.465861 (+ 280us) server_negotiation.cc:407] Connection header received
0812 16:34:44.466398 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:44.466400 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:44.466412 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:44.466462 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:44.467124 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:44.467913 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:44.469093 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:44.469344 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:44.469425 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:44.471600 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:44.471604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:44.471604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:44.471617 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:44.473124 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:44.473567 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:44.473569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:44.473569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:44.473632 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:44.473908 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:44.473909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:44.473910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:44.474033 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:44.474112 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:44.474480 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 16:34:44.474564 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 16:34:44.474825 2473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:44.465523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:45001 (local address 127.19.238.67:33349)
0812 16:34:44.465707 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:44.465750 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:44.466285 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:44.466531 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:44.466535 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:44.466751 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:44.466997 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:44.467003 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:44.468030 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:44.468032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:44.468995 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:44.468998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:44.469456 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:44.469458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:44.469581 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:44.470154 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:44.470163 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:44.471491 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:44.473228 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:44.473237 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:44.473239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:44.473486 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:44.473703 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:44.473704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:44.473704 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:44.473830 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:44.474142 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:44.474148 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:44.474289 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 16:34:44.474400 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 16:34:44.474571 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":108,"threads_started":1}
I20260812 16:34:44.475368 2470 heartbeater.cc:347] Connected to a master server at 127.19.238.126:45001
I20260812 16:34:44.475500 2470 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:44.475699 2470 heartbeater.cc:511] Master 127.19.238.126:45001 requested a full tablet report, sending...
I20260812 16:34:44.476141 1944 ts_manager.cc:194] Registered new tserver with Master: c1126bc5d2e84fc1be4b8012ae372a79 (127.19.238.67:45927)
I20260812 16:34:44.476536 1944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:33349
I20260812 16:34:44.479163 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:34:44.479895 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 2031
I20260812 16:34:44.495393 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 2181
I20260812 16:34:44.502768 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 2329
I20260812 16:34:44.517706 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 1911
I20260812 16:34:44.530192 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:45001
--webserver_interface=127.19.238.126
--webserver_port=39103
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:45001
--ranger_config_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.238.84:56503/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.238.84:56503/kms with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:34:44.615208 2474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:44.615450 2474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:44.615474 2474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:44.616849 2474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:34:44.616905 2474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:44.616920 2474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:44.616932 2474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:34:44.616946 2474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:34:44.618688 2474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.238.84:56503/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:45001
--ranger_config_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:45001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=39103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:44.618958 2474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:44.619235 2474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:44.624920 2484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:44.624920 2481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:44.625352 2480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:44.625591 2474 server_base.cc:1034] running on GCE node
I20260812 16:34:44.625900 2474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:44.626212 2474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:44.627368 2474 hybrid_clock.cc:648] HybridClock initialized: now 1786552484627351 us; error 31 us; skew 500 ppm
Aug 12 16:34:44 dist-test-slave-mvvj krb5kdc[1414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552484, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:44.630261 2474 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:34:44.630782 2474 webserver.cc:460] Webserver started at http://127.19.238.126:39103/ using document root <none> and password file <none>
I20260812 16:34:44.631047 2474 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:44.631104 2474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 16:34:44 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552484, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
I20260812 16:34:44.713353 2474 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:34:44.714268 2495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:44.714478 2474 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:34:44.714562 2474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "7640a127d2f44227b38cd312c486a133"
format_stamp: "Formatted at 2026-08-12 16:34:42 on dist-test-slave-mvvj"
server_key: "9d8f94957faa472f066746339ee973bc"
server_key_iv: "21d57d8f0a1ec75f5471de207eef0c95"
server_key_version: "kuduclusterkey@0"
I20260812 16:34:44.714974 2474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:44.730038 2474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:44.732051 2474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:44.732182 2474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:44.736905 2474 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:45001
I20260812 16:34:44.736981 2552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:45001 every 8 connection(s)
I20260812 16:34:44.737393 2474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:34:44.738754 2553 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:34:44.739439 2553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133: Bootstrap starting.
I20260812 16:34:44.747520 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 2474
I20260812 16:34:44.747903 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.238.84:56503/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:39681
--local_ip_for_outbound_sockets=127.19.238.65
--tserver_master_addrs=127.19.238.126:45001
--webserver_port=45731
--webserver_interface=127.19.238.65
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.238.84:56503/kms with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:34:44.747915 2553 log.cc:824] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133: Log is configured to *not* fsync() on all Append() calls
I20260812 16:34:44.750391 2553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:34:44.750777 2553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133: Bootstrap complete.
I20260812 16:34:44.755085 2553 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7640a127d2f44227b38cd312c486a133" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45001 } }
I20260812 16:34:44.755528 2553 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7640a127d2f44227b38cd312c486a133, State: Initialized, Role: FOLLOWER
I20260812 16:34:44.755729 2553 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7640a127d2f44227b38cd312c486a133" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45001 } }
I20260812 16:34:44.755856 2553 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:34:44.755921 2553 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:34:44.755983 2553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [term 1 FOLLOWER]: Advancing to term 2
I20260812 16:34:44.757759 2553 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7640a127d2f44227b38cd312c486a133" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45001 } }
I20260812 16:34:44.757915 2553 leader_election.cc:302] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7640a127d2f44227b38cd312c486a133; no voters:
I20260812 16:34:44.758117 2553 leader_election.cc:288] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 16:34:44.758208 2558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 16:34:44.758453 2558 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [term 2 LEADER]: Becoming Leader. State: Replica: 7640a127d2f44227b38cd312c486a133, State: Running, Role: LEADER
I20260812 16:34:44.758461 2553 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:34:44.758644 2558 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7640a127d2f44227b38cd312c486a133" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45001 } }
I20260812 16:34:44.759073 2559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7640a127d2f44227b38cd312c486a133" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7640a127d2f44227b38cd312c486a133" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45001 } } }
I20260812 16:34:44.759172 2559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [sys.catalog]: This master's current role is: LEADER
I20260812 16:34:44.759085 2561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7640a127d2f44227b38cd312c486a133. Latest consensus state: current_term: 2 leader_uuid: "7640a127d2f44227b38cd312c486a133" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7640a127d2f44227b38cd312c486a133" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45001 } } }
I20260812 16:34:44.759248 2561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133 [sys.catalog]: This master's current role is: LEADER
I20260812 16:34:44.759934 2566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:34:44.760488 2566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:34:44.760627 2566 catalog_manager.cc:1259] Loaded cluster ID: f107178afb104e45ba3ace6ab7265f14
I20260812 16:34:44.760658 2566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:34:44.760870 2553 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 16:34:44.761641 2566 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:34:44.761785 2566 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133: Loaded TSK: 0
I20260812 16:34:44.761929 2566 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 16:34:44.870563 2557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:44.871059 2557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:44.871127 2557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:44.873124 2557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:44.873200 2557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:44.873246 2557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:34:44.875674 2557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.238.84:56503/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:39681
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=45731
--tserver_master_addrs=127.19.238.126:45001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:44.876011 2557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:44.876343 2557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:44.883311 2596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:44.883327 2593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:44.883627 2594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:44.884065 2557 server_base.cc:1034] running on GCE node
I20260812 16:34:44.884301 2557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:44.884604 2557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:44.887882 2557 hybrid_clock.cc:648] HybridClock initialized: now 1786552484887827 us; error 60 us; skew 500 ppm
Aug 12 16:34:44 dist-test-slave-mvvj krb5kdc[1414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552484, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:44.891629 2557 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:34:44.892302 2557 webserver.cc:460] Webserver started at http://127.19.238.65:45731/ using document root <none> and password file <none>
I20260812 16:34:44.892624 2557 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:44.892733 2557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 16:34:44 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552484, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
I20260812 16:34:44.924678 2557 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.001s sys 0.000s
I20260812 16:34:44.926501 2604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:44.926743 2557 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260812 16:34:44.926839 2557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "5db58a364a034627a46072903439e7cb"
format_stamp: "Formatted at 2026-08-12 16:34:43 on dist-test-slave-mvvj"
server_key: "fa8af2e9a2d16d367b9a27521be92a20"
server_key_iv: "5960d95eadfe71de1600b15d93ecc5b9"
server_key_version: "kuduclusterkey@0"
I20260812 16:34:44.927371 2557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:44.950017 2557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:44.952805 2557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:44.953053 2557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:44.953406 2557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:34:44.953897 2557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:34:44.954002 2557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:44.954149 2557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:34:44.954231 2557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:44.961509 2557 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:39681
I20260812 16:34:44.962201 2557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:34:44.962165 2727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:39681 every 8 connection(s)
I20260812 16:34:44.967986 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 2557
I20260812 16:34:44.968756 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.238.84:56503/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:43033
--local_ip_for_outbound_sockets=127.19.238.66
--tserver_master_addrs=127.19.238.126:45001
--webserver_port=42083
--webserver_interface=127.19.238.66
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.238.84:56503/kms with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:34:44 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552484, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:34:44.979434 2731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:44.963332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:44979 (local address 127.19.238.126:45001)
0812 16:34:44.963549 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:44.963559 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:44.963785 (+ 226us) server_negotiation.cc:407] Connection header received
0812 16:34:44.964496 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:44.964500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:44.964549 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:44.964697 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:44.965299 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:44.966182 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:44.967210 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:44.967632 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:44.967757 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:44.974573 (+ 6816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:44.974580 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:44.974581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:44.974598 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:44.977144 (+ 2546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:44.977815 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:44.977827 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:44.977828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:44.977964 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:44.978298 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:44.978299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:44.978311 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:44.978593 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:44.978708 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:44.979099 (+ 391us) server_negotiation.cc:299] Negotiation successful
0812 16:34:44.979252 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":76,"threads_started":1}
I20260812 16:34:44.980113 2730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:44.963273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:45001 (local address 127.19.238.65:44979)
0812 16:34:44.963637 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:44.963672 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:44.964239 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:44.964742 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:44.964747 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:44.964977 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:44.965174 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:44.965179 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:44.966306 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:44.966307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:44.967095 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:44.967097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:44.967761 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:44.967762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:44.967949 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:44.970652 (+ 2703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:44.970664 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:44.974355 (+ 3691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:44.977284 (+ 2929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:44.977296 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:44.977299 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:44.977692 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:44.978075 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:44.978075 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:44.978076 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:44.978211 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:44.978685 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:44.978693 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:44.978882 (+ 189us) client_negotiation.cc:769] Sending connection context
0812 16:34:44.979668 (+ 786us) client_negotiation.cc:240] Negotiation successful
0812 16:34:44.979881 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":114,"threads_started":1}
I20260812 16:34:44.981189 2728 heartbeater.cc:347] Connected to a master server at 127.19.238.126:45001
I20260812 16:34:44.981292 2728 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:44.981496 2728 heartbeater.cc:511] Master 127.19.238.126:45001 requested a full tablet report, sending...
W20260812 16:34:44.982481 2728 heartbeater.cc:644] Failed to heartbeat to 127.19.238.126:45001 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 16:34:44.982554 2728 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:44.982620 2728 heartbeater.cc:511] Master 127.19.238.126:45001 requested a full tablet report, sending...
I20260812 16:34:44.983085 2728 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:44.983186 2728 heartbeater.cc:511] Master 127.19.238.126:45001 requested a full tablet report, sending...
W20260812 16:34:44.983731 2728 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 16:34:45.098840 2732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:45.099180 2732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:45.099221 2732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:45.101266 2732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:45.101408 2732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:45.101476 2732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:34:45.104166 2732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.238.84:56503/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:43033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=42083
--tserver_master_addrs=127.19.238.126:45001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:45.104512 2732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:45.104835 2732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:45.111676 2740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:45.111716 2737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:45.112526 2732 server_base.cc:1034] running on GCE node
W20260812 16:34:45.113808 2738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:45.114262 2732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:45.114602 2732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:45.115780 2732 hybrid_clock.cc:648] HybridClock initialized: now 1786552485115768 us; error 31 us; skew 500 ppm
Aug 12 16:34:45 dist-test-slave-mvvj krb5kdc[1414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552485, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:45.119453 2732 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:34:45.120146 2732 webserver.cc:460] Webserver started at http://127.19.238.66:42083/ using document root <none> and password file <none>
I20260812 16:34:45.120467 2732 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:45.120536 2732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 16:34:45 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552485, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
I20260812 16:34:45.140333 2732 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:34:45.141264 2748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:45.141458 2732 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:34:45.141554 2732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "3e9381d6c94f42d38b0c9d4d5ef058a3"
format_stamp: "Formatted at 2026-08-12 16:34:44 on dist-test-slave-mvvj"
server_key: "27c3895b8a12edf04e0fd5462b50d9a9"
server_key_iv: "91f29dd67af69ac3ffc1b78e16f8a94f"
server_key_version: "kuduclusterkey@0"
I20260812 16:34:45.141958 2732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:45.158384 2732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:45.160737 2732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:45.160884 2732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:45.161201 2732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:34:45.161657 2732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:34:45.161698 2732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:45.161741 2732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:34:45.161770 2732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:45.169471 2732 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:43033
I20260812 16:34:45.184032 2872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:43033 every 8 connection(s)
I20260812 16:34:45.184280 2732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:34:45.185097 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 2732
I20260812 16:34:45.185432 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.238.84:56503/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:45927
--local_ip_for_outbound_sockets=127.19.238.67
--tserver_master_addrs=127.19.238.126:45001
--webserver_port=43443
--webserver_interface=127.19.238.67
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.238.84:56503/kms with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:34:45 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552485, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:34:45.196756 2731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:45.185393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:34887 (local address 127.19.238.126:45001)
0812 16:34:45.185443 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:45.185446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:45.185930 (+ 484us) server_negotiation.cc:407] Connection header received
0812 16:34:45.186768 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:45.186770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:45.186786 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:45.186839 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:45.187771 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:45.188611 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:45.189759 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:45.190092 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:45.190166 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:45.192983 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:45.192985 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:45.192985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:45.192994 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:45.194817 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:45.195455 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:45.195456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:45.195457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:45.195526 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:45.195893 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:45.195894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:45.195895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:45.196051 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:45.196101 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:45.196552 (+ 451us) server_negotiation.cc:299] Negotiation successful
0812 16:34:45.196654 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:34:45.197368 2876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:45.185219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:45001 (local address 127.19.238.66:34887)
0812 16:34:45.185771 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:45.185824 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:45.186628 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:45.187048 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:45.187052 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:45.187336 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:45.187641 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:45.187650 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:45.188748 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:45.188750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:45.189651 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:45.189654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:45.190264 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:45.190266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:45.190433 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:45.191104 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:45.191123 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:45.192823 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:45.194961 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:45.194971 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:45.194974 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:45.195326 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:45.195616 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:45.195616 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:45.195617 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:45.195770 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:45.196156 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:45.196163 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:45.196349 (+ 186us) client_negotiation.cc:769] Sending connection context
0812 16:34:45.196921 (+ 572us) client_negotiation.cc:240] Negotiation successful
0812 16:34:45.197115 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":94,"threads_started":1}
I20260812 16:34:45.197887 2873 heartbeater.cc:347] Connected to a master server at 127.19.238.126:45001
I20260812 16:34:45.198004 2873 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:45.198235 2873 heartbeater.cc:511] Master 127.19.238.126:45001 requested a full tablet report, sending...
W20260812 16:34:45.198895 2873 heartbeater.cc:644] Failed to heartbeat to 127.19.238.126:45001 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 16:34:45.198978 2873 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:45.199056 2873 heartbeater.cc:511] Master 127.19.238.126:45001 requested a full tablet report, sending...
I20260812 16:34:45.199379 2873 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:45.199476 2873 heartbeater.cc:511] Master 127.19.238.126:45001 requested a full tablet report, sending...
W20260812 16:34:45.199792 2873 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 16:34:45.300251 2877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:45.300499 2877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:45.300542 2877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:45.302125 2877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:45.302196 2877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:45.302240 2877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:34:45.304478 2877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.238.84:56503/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:45927
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=43443
--tserver_master_addrs=127.19.238.126:45001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:45.304756 2877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:45.305056 2877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:45.311033 2884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:45.311256 2877 server_base.cc:1034] running on GCE node
W20260812 16:34:45.311018 2887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:45.311018 2885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:45.311718 2877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:45.312034 2877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:45.313198 2877 hybrid_clock.cc:648] HybridClock initialized: now 1786552485313187 us; error 26 us; skew 500 ppm
Aug 12 16:34:45 dist-test-slave-mvvj krb5kdc[1414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552485, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:45.316042 2877 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:34:45.316567 2877 webserver.cc:460] Webserver started at http://127.19.238.67:43443/ using document root <none> and password file <none>
I20260812 16:34:45.316821 2877 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:45.316903 2877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 16:34:45 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552485, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
I20260812 16:34:45.348347 2877 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260812 16:34:45.352695 2896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:45.356918 2877 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260812 16:34:45.357074 2877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "c1126bc5d2e84fc1be4b8012ae372a79"
format_stamp: "Formatted at 2026-08-12 16:34:44 on dist-test-slave-mvvj"
server_key: "e99a84f60acc8bfca6f98a371c7d6a68"
server_key_iv: "170cf4ed61b9622e2910b589b0858869"
server_key_version: "kuduclusterkey@0"
I20260812 16:34:45.357623 2877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:45.385082 2877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:45.387728 2877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:45.387934 2877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:45.388236 2877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:34:45.388615 2877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:34:45.388661 2877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:45.388693 2877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:34:45.388718 2877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:45.396523 2877 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:45927
I20260812 16:34:45.396963 2877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:34:45.397641 3015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:45927 every 8 connection(s)
I20260812 16:34:45.399801 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 2877
Aug 12 16:34:45 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552485, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:34:45.408499 3019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:45.400305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56682 (local address 127.19.238.126:45001)
0812 16:34:45.400486 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:45.400488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:45.400535 (+ 47us) server_negotiation.cc:407] Connection header received
0812 16:34:45.400617 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:45.400617 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:45.400630 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:45.400691 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:45.401755 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:45.402583 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:45.403395 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:45.403714 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:45.403794 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:45.405237 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:45.405239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:45.405239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:45.405247 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:45.407190 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:45.407591 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:45.407593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:45.407593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:45.407659 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:45.407906 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:45.407907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:45.407907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:45.408054 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:45.408238 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:45.408280 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 16:34:45.408383 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":152,"thread_start_us":66,"threads_started":1}
I20260812 16:34:45.409063 2731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:45.397847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:36765 (local address 127.19.238.126:45001)
0812 16:34:45.397909 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:45.397912 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:45.398140 (+ 228us) server_negotiation.cc:407] Connection header received
0812 16:34:45.398760 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:45.398762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:45.398775 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:45.398828 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:45.399461 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:45.400093 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:45.401717 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:45.402021 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:45.402103 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:45.405362 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:45.405363 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:45.405364 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:45.405371 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:45.407146 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:45.407669 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:45.407670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:45.407670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:45.407728 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:45.408054 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:45.408055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:45.408055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:45.408183 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:45.408246 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:45.408887 (+ 641us) server_negotiation.cc:299] Negotiation successful
0812 16:34:45.408972 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 16:34:45.409317 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:45.397847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:45001 (local address 127.19.238.67:36765)
0812 16:34:45.398006 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:45.398049 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:45.398617 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:45.398900 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:45.398904 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:45.399126 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:45.399329 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:45.399333 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:45.400208 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:45.400210 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:45.401601 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:45.401604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:45.402149 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:45.402151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:45.402278 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:45.403449 (+ 1171us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:45.403459 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:45.405257 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:45.407254 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:45.407264 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:45.407267 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:45.407604 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:45.407801 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:45.407802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:45.407802 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:45.407972 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:45.408275 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:45.408281 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:45.408459 (+ 178us) client_negotiation.cc:769] Sending connection context
0812 16:34:45.408865 (+ 406us) client_negotiation.cc:240] Negotiation successful
0812 16:34:45.409047 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260812 16:34:45.409968 3016 heartbeater.cc:347] Connected to a master server at 127.19.238.126:45001
I20260812 16:34:45.410065 3016 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:45.410247 3016 heartbeater.cc:511] Master 127.19.238.126:45001 requested a full tablet report, sending...
W20260812 16:34:45.410894 3016 heartbeater.cc:644] Failed to heartbeat to 127.19.238.126:45001 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 16:34:45.410986 3016 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:45.411059 3016 heartbeater.cc:511] Master 127.19.238.126:45001 requested a full tablet report, sending...
I20260812 16:34:45.411384 3016 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:45.411463 3016 heartbeater.cc:511] Master 127.19.238.126:45001 requested a full tablet report, sending...
W20260812 16:34:45.411788 3016 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 16:34:45 dist-test-slave-mvvj krb5kdc[1414](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786552485, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:45.983945 2728 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:45.984145 2728 heartbeater.cc:511] Master 127.19.238.126:45001 requested a full tablet report, sending...
I20260812 16:34:45.984598 2517 ts_manager.cc:194] Registered new tserver with Master: 5db58a364a034627a46072903439e7cb (127.19.238.65:39681)
I20260812 16:34:45.985630 2517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:44979
I20260812 16:34:46.199965 2873 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:46.200156 2873 heartbeater.cc:511] Master 127.19.238.126:45001 requested a full tablet report, sending...
I20260812 16:34:46.200547 2517 ts_manager.cc:194] Registered new tserver with Master: 3e9381d6c94f42d38b0c9d4d5ef058a3 (127.19.238.66:43033)
I20260812 16:34:46.201020 2517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:34887
Aug 12 16:34:46 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552485, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
I20260812 16:34:46.412009 3016 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:46.412175 3016 heartbeater.cc:511] Master 127.19.238.126:45001 requested a full tablet report, sending...
I20260812 16:34:46.412598 2517 ts_manager.cc:194] Registered new tserver with Master: c1126bc5d2e84fc1be4b8012ae372a79 (127.19.238.67:45927)
I20260812 16:34:46.413168 2517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:36765
I20260812 16:34:46.423687 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:34:46.433440 3038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:46.424809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39936 (local address 127.19.238.126:45001)
0812 16:34:46.425168 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:46.425170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:46.425284 (+ 114us) server_negotiation.cc:407] Connection header received
0812 16:34:46.425382 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:46.425383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:46.425397 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:46.425441 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:46.426186 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.426988 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.428211 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.428528 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.428570 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:46.430108 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:46.430110 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:46.430110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:46.430117 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:46.432082 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:46.432483 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:46.432484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:46.432485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:46.432563 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:46.432765 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:46.432765 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:46.432766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:46.432895 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:46.433000 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:46.433242 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 16:34:46.433329 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":75,"threads_started":1}
I20260812 16:34:46.436583 2517 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39936:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:34:46.438841 2517 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:34:46.457134 3047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:46.448386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:39681 (local address 127.0.0.1:45254)
0812 16:34:46.451588 (+ 3202us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:46.451596 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:46.451630 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:46.452187 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:46.452191 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:46.452193 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:34:46.452270 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:46.452272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.453960 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.453961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:46.455300 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:46.455304 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.456767 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.456768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:46.456864 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:46.456879 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 16:34:46.456968 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 16:34:46.457027 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3188,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260812 16:34:46.457732 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:46.448837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45254 (local address 127.19.238.65:39681)
0812 16:34:46.451891 (+ 3054us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:46.451897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:46.451908 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:34:46.451954 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:46.451957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:46.451975 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:46.452067 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:34:46.452349 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.453861 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.455414 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.456593 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.456665 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:46.456728 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:46.457598 (+ 870us) server_negotiation.cc:299] Negotiation successful
0812 16:34:46.457649 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3015,"thread_start_us":78,"threads_started":1}
I20260812 16:34:46.458717 2648 tablet_service.cc:1467] Processing CreateTablet for tablet aa8cafd0ff4540f5a8e0fc16cc2748e0 (DEFAULT_TABLE table=test-table [id=15ee4ddfeb434ee383985d1716a2cb59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:34:46.459167 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa8cafd0ff4540f5a8e0fc16cc2748e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:34:46.462757 3046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:46.448386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:43033 (local address 127.0.0.1:37828)
0812 16:34:46.451364 (+ 2978us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:46.451418 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:46.451484 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:46.458351 (+ 6867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:46.458352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:46.458355 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:34:46.458404 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:46.458405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.460288 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.460289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:46.461444 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:46.461447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.462514 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.462515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:46.462584 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:46.462589 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:34:46.462620 (+ 31us) client_negotiation.cc:240] Negotiation successful
0812 16:34:46.462659 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2940,"thread_start_us":97,"threads_started":1}
I20260812 16:34:46.463304 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:46.448668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37828 (local address 127.19.238.66:43033)
0812 16:34:46.458106 (+ 9438us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:46.458112 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:46.458125 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:34:46.458166 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:46.458168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:46.458186 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:46.458478 (+ 292us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:34:46.458573 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.460189 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.461561 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.462410 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.463020 (+ 610us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:46.463095 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:46.463124 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 16:34:46.463165 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9393,"thread_start_us":61,"threads_started":1}
I20260812 16:34:46.464123 2791 tablet_service.cc:1467] Processing CreateTablet for tablet aa8cafd0ff4540f5a8e0fc16cc2748e0 (DEFAULT_TABLE table=test-table [id=15ee4ddfeb434ee383985d1716a2cb59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:34:46.464587 2791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa8cafd0ff4540f5a8e0fc16cc2748e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:34:46.468168 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:46.448626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:45927 (local address 127.0.0.1:43500)
0812 16:34:46.451720 (+ 3094us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:46.451727 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:46.451760 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:46.464357 (+ 12597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:46.464358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:46.464361 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:34:46.464410 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:46.464412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.465739 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.465740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:46.466882 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:46.466885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.467909 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.467910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:46.468007 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:46.468013 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 16:34:46.468044 (+ 31us) client_negotiation.cc:240] Negotiation successful
0812 16:34:46.468086 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3081,"thread_start_us":62,"threads_started":1}
I20260812 16:34:46.468559 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:46.448932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43500 (local address 127.19.238.67:45927)
0812 16:34:46.464161 (+ 15229us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:46.464166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:46.464178 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:34:46.464216 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:46.464218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:46.464235 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:46.464482 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:34:46.464574 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.465635 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.467004 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.467772 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.468337 (+ 565us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:46.468411 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:46.468439 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 16:34:46.468480 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15168,"thread_start_us":73,"threads_started":1}
I20260812 16:34:46.469218 2946 tablet_service.cc:1467] Processing CreateTablet for tablet aa8cafd0ff4540f5a8e0fc16cc2748e0 (DEFAULT_TABLE table=test-table [id=15ee4ddfeb434ee383985d1716a2cb59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:34:46.469553 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa8cafd0ff4540f5a8e0fc16cc2748e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:34:46.470168 3052 tablet_bootstrap.cc:492] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3: Bootstrap starting.
I20260812 16:34:46.471010 3052 tablet_bootstrap.cc:654] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3: Neither blocks nor log segments found. Creating new log.
I20260812 16:34:46.471453 3052 log.cc:824] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3: Log is configured to *not* fsync() on all Append() calls
I20260812 16:34:46.472286 3052 tablet_bootstrap.cc:492] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3: No bootstrap required, opened a new log
I20260812 16:34:46.472360 3052 ts_tablet_manager.cc:1397] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:34:46.475003 3052 raft_consensus.cc:348] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9381d6c94f42d38b0c9d4d5ef058a3" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43033 } } peers { permanent_uuid: "5db58a364a034627a46072903439e7cb" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39681 } } peers { permanent_uuid: "c1126bc5d2e84fc1be4b8012ae372a79" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45927 } }
I20260812 16:34:46.475176 3052 raft_consensus.cc:374] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:34:46.475214 3052 raft_consensus.cc:729] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9381d6c94f42d38b0c9d4d5ef058a3, State: Initialized, Role: FOLLOWER
I20260812 16:34:46.475963 3052 consensus_queue.cc:260] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9381d6c94f42d38b0c9d4d5ef058a3" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43033 } } peers { permanent_uuid: "5db58a364a034627a46072903439e7cb" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39681 } } peers { permanent_uuid: "c1126bc5d2e84fc1be4b8012ae372a79" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45927 } }
I20260812 16:34:46.476388 3052 ts_tablet_manager.cc:1428] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 16:34:46.476572 2873 heartbeater.cc:503] Master 127.19.238.126:45001 was elected leader, sending a full tablet report...
I20260812 16:34:46.477629 3055 tablet_bootstrap.cc:492] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79: Bootstrap starting.
I20260812 16:34:46.478463 3055 tablet_bootstrap.cc:654] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79: Neither blocks nor log segments found. Creating new log.
I20260812 16:34:46.479353 3055 log.cc:824] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79: Log is configured to *not* fsync() on all Append() calls
I20260812 16:34:46.480257 3055 tablet_bootstrap.cc:492] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79: No bootstrap required, opened a new log
I20260812 16:34:46.480327 3055 ts_tablet_manager.cc:1397] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:34:46.482583 3055 raft_consensus.cc:348] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9381d6c94f42d38b0c9d4d5ef058a3" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43033 } } peers { permanent_uuid: "5db58a364a034627a46072903439e7cb" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39681 } } peers { permanent_uuid: "c1126bc5d2e84fc1be4b8012ae372a79" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45927 } }
I20260812 16:34:46.482748 3055 raft_consensus.cc:374] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:34:46.482780 3055 raft_consensus.cc:729] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1126bc5d2e84fc1be4b8012ae372a79, State: Initialized, Role: FOLLOWER
I20260812 16:34:46.482913 3055 consensus_queue.cc:260] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9381d6c94f42d38b0c9d4d5ef058a3" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43033 } } peers { permanent_uuid: "5db58a364a034627a46072903439e7cb" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39681 } } peers { permanent_uuid: "c1126bc5d2e84fc1be4b8012ae372a79" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45927 } }
I20260812 16:34:46.483171 3055 ts_tablet_manager.cc:1428] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:34:46.483912 3016 heartbeater.cc:503] Master 127.19.238.126:45001 was elected leader, sending a full tablet report...
I20260812 16:34:46.488106 3053 tablet_bootstrap.cc:492] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 5db58a364a034627a46072903439e7cb: Bootstrap starting.
I20260812 16:34:46.489019 3053 tablet_bootstrap.cc:654] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 5db58a364a034627a46072903439e7cb: Neither blocks nor log segments found. Creating new log.
I20260812 16:34:46.489470 3053 log.cc:824] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 5db58a364a034627a46072903439e7cb: Log is configured to *not* fsync() on all Append() calls
I20260812 16:34:46.490459 3053 tablet_bootstrap.cc:492] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 5db58a364a034627a46072903439e7cb: No bootstrap required, opened a new log
I20260812 16:34:46.490535 3053 ts_tablet_manager.cc:1397] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 5db58a364a034627a46072903439e7cb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:34:46.493090 3053 raft_consensus.cc:348] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 5db58a364a034627a46072903439e7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9381d6c94f42d38b0c9d4d5ef058a3" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43033 } } peers { permanent_uuid: "5db58a364a034627a46072903439e7cb" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39681 } } peers { permanent_uuid: "c1126bc5d2e84fc1be4b8012ae372a79" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45927 } }
I20260812 16:34:46.493273 3053 raft_consensus.cc:374] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 5db58a364a034627a46072903439e7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:34:46.493309 3053 raft_consensus.cc:729] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 5db58a364a034627a46072903439e7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db58a364a034627a46072903439e7cb, State: Initialized, Role: FOLLOWER
I20260812 16:34:46.494021 3053 consensus_queue.cc:260] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 5db58a364a034627a46072903439e7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9381d6c94f42d38b0c9d4d5ef058a3" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43033 } } peers { permanent_uuid: "5db58a364a034627a46072903439e7cb" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39681 } } peers { permanent_uuid: "c1126bc5d2e84fc1be4b8012ae372a79" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45927 } }
I20260812 16:34:46.494397 3053 ts_tablet_manager.cc:1428] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 5db58a364a034627a46072903439e7cb: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 16:34:46.494580 2728 heartbeater.cc:503] Master 127.19.238.126:45001 was elected leader, sending a full tablet report...
I20260812 16:34:46.620843 3056 raft_consensus.cc:482] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:34:46.621007 3056 raft_consensus.cc:504] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9381d6c94f42d38b0c9d4d5ef058a3" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43033 } } peers { permanent_uuid: "5db58a364a034627a46072903439e7cb" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39681 } } peers { permanent_uuid: "c1126bc5d2e84fc1be4b8012ae372a79" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45927 } }
I20260812 16:34:46.621425 3056 leader_election.cc:288] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5db58a364a034627a46072903439e7cb (127.19.238.65:39681), c1126bc5d2e84fc1be4b8012ae372a79 (127.19.238.67:45927)
Aug 12 16:34:46 dist-test-slave-mvvj krb5kdc[1414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552485, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for HTTP/127.19.238.84@KRBTEST.COM
I20260812 16:34:46.641271 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:46.621724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:39681 (local address 127.19.238.66:58133)
0812 16:34:46.628851 (+ 7127us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:46.630007 (+ 1156us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:46.630150 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:46.630356 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:46.630357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:46.630359 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:34:46.630451 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:46.630453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.632697 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.632698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:46.637259 (+ 4561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:46.637267 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.640965 (+ 3698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.640966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:46.641040 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:46.641045 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:34:46.641103 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 16:34:46.641148 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7095,"thread_start_us":95,"threads_started":1}
I20260812 16:34:46.641271 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:46.623035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:58133 (local address 127.19.238.65:39681)
0812 16:34:46.623092 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:46.623094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:46.630103 (+ 7009us) server_negotiation.cc:407] Connection header received
0812 16:34:46.630239 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:46.630240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:46.630251 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:46.630283 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:34:46.630528 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.631285 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.639956 (+ 8671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.640531 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.640586 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:46.640596 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:46.641140 (+ 544us) server_negotiation.cc:299] Negotiation successful
0812 16:34:46.641199 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 16:34:46.643177 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa8cafd0ff4540f5a8e0fc16cc2748e0" candidate_uuid: "3e9381d6c94f42d38b0c9d4d5ef058a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db58a364a034627a46072903439e7cb" is_pre_election: true
I20260812 16:34:46.643391 2674 raft_consensus.cc:2454] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 5db58a364a034627a46072903439e7cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9381d6c94f42d38b0c9d4d5ef058a3 in term 0.
I20260812 16:34:46.643699 2751 leader_election.cc:302] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9381d6c94f42d38b0c9d4d5ef058a3, 5db58a364a034627a46072903439e7cb; no voters:
I20260812 16:34:46.643875 3056 raft_consensus.cc:2781] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:34:46.643936 3056 raft_consensus.cc:482] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:34:46.643961 3056 raft_consensus.cc:3037] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:34:46.645030 3056 raft_consensus.cc:504] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9381d6c94f42d38b0c9d4d5ef058a3" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43033 } } peers { permanent_uuid: "5db58a364a034627a46072903439e7cb" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39681 } } peers { permanent_uuid: "c1126bc5d2e84fc1be4b8012ae372a79" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45927 } }
I20260812 16:34:46.645174 3056 leader_election.cc:288] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [CANDIDATE]: Term 1 election: Requested vote from peers 5db58a364a034627a46072903439e7cb (127.19.238.65:39681), c1126bc5d2e84fc1be4b8012ae372a79 (127.19.238.67:45927)
I20260812 16:34:46.645561 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa8cafd0ff4540f5a8e0fc16cc2748e0" candidate_uuid: "3e9381d6c94f42d38b0c9d4d5ef058a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db58a364a034627a46072903439e7cb"
I20260812 16:34:46.645650 2674 raft_consensus.cc:3037] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 5db58a364a034627a46072903439e7cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:34:46.646597 2674 raft_consensus.cc:2454] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 5db58a364a034627a46072903439e7cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9381d6c94f42d38b0c9d4d5ef058a3 in term 1.
I20260812 16:34:46.646826 2751 leader_election.cc:302] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9381d6c94f42d38b0c9d4d5ef058a3, 5db58a364a034627a46072903439e7cb; no voters:
I20260812 16:34:46.646981 3056 raft_consensus.cc:2781] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:34:46.647166 3056 raft_consensus.cc:686] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [term 1 LEADER]: Becoming Leader. State: Replica: 3e9381d6c94f42d38b0c9d4d5ef058a3, State: Running, Role: LEADER
I20260812 16:34:46.647271 3056 consensus_queue.cc:237] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9381d6c94f42d38b0c9d4d5ef058a3" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43033 } } peers { permanent_uuid: "5db58a364a034627a46072903439e7cb" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39681 } } peers { permanent_uuid: "c1126bc5d2e84fc1be4b8012ae372a79" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45927 } }
W20260812 16:34:46.648380 3017 tablet.cc:2369] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:34:46.648280 2517 catalog_manager.cc:5495] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e9381d6c94f42d38b0c9d4d5ef058a3 (127.19.238.66). New cstate: current_term: 1 leader_uuid: "3e9381d6c94f42d38b0c9d4d5ef058a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e9381d6c94f42d38b0c9d4d5ef058a3" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 43033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db58a364a034627a46072903439e7cb" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1126bc5d2e84fc1be4b8012ae372a79" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45927 } health_report { overall_health: UNKNOWN } } }
I20260812 16:34:46.650108 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:46.622287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:35649 (local address 127.19.238.67:45927)
0812 16:34:46.622339 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:46.622341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:46.637496 (+ 15155us) server_negotiation.cc:407] Connection header received
0812 16:34:46.637611 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:46.637613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:46.637623 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:46.637654 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:34:46.638392 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.639120 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.648884 (+ 9764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.649504 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.649564 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:46.649574 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:46.649992 (+ 418us) server_negotiation.cc:299] Negotiation successful
0812 16:34:46.650044 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 16:34:46.650117 3062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:46.621931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:45927 (local address 127.19.238.66:35649)
0812 16:34:46.637426 (+ 15495us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:46.637439 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:46.637532 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:46.638183 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:46.638185 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:46.638188 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:34:46.638263 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:46.638266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.639221 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.639222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:46.642564 (+ 3342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:46.642568 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.649741 (+ 7173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.649742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:46.649824 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:46.649888 (+ 64us) client_negotiation.cc:769] Sending connection context
0812 16:34:46.649973 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 16:34:46.650025 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15470,"thread_start_us":79,"threads_started":1}
I20260812 16:34:46.650483 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa8cafd0ff4540f5a8e0fc16cc2748e0" candidate_uuid: "3e9381d6c94f42d38b0c9d4d5ef058a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1126bc5d2e84fc1be4b8012ae372a79" is_pre_election: true
I20260812 16:34:46.650633 2968 raft_consensus.cc:2454] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9381d6c94f42d38b0c9d4d5ef058a3 in term 0.
I20260812 16:34:46.650758 2967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa8cafd0ff4540f5a8e0fc16cc2748e0" candidate_uuid: "3e9381d6c94f42d38b0c9d4d5ef058a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1126bc5d2e84fc1be4b8012ae372a79"
I20260812 16:34:46.650821 2967 raft_consensus.cc:3037] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:34:46.651755 2967 raft_consensus.cc:2454] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9381d6c94f42d38b0c9d4d5ef058a3 in term 1.
W20260812 16:34:46.686333 2875 tablet.cc:2369] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:34:46.693627 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:46.690141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37840 (local address 127.19.238.66:43033)
0812 16:34:46.690183 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:46.690185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:46.690193 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:34:46.690238 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:46.690239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:46.690262 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:46.690330 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:34:46.690599 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.691342 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.692478 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:46.692778 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:46.692841 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:46.693061 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:34:46.693202 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:34:46.693281 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:46.693522 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 16:34:46.693545 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 16:34:46.701234 2967 raft_consensus.cc:1261] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79 [term 1 FOLLOWER]: Refusing update from remote peer 3e9381d6c94f42d38b0c9d4d5ef058a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:34:46.701737 3064 consensus_queue.cc:1035] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1126bc5d2e84fc1be4b8012ae372a79" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:34:46.711588 2674 raft_consensus.cc:1261] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 5db58a364a034627a46072903439e7cb [term 1 FOLLOWER]: Refusing update from remote peer 3e9381d6c94f42d38b0c9d4d5ef058a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:34:46.712122 3064 consensus_queue.cc:1035] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db58a364a034627a46072903439e7cb" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 16:34:46.714258 2729 tablet.cc:2369] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 5db58a364a034627a46072903439e7cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:34:46.717442 3068 mvcc.cc:205] Tried to move back new op lower bound from 7317718985524355072 to 7317718985307619328. Current Snapshot: MvccSnapshot[applied={T|T < 7317718985524355072}]
I20260812 16:34:46.717756 3071 mvcc.cc:205] Tried to move back new op lower bound from 7317718985524355072 to 7317718985307619328. Current Snapshot: MvccSnapshot[applied={T|T < 7317718985524355072}]
I20260812 16:34:46.724145 3069 mvcc.cc:205] Tried to move back new op lower bound from 7317718985524355072 to 7317718985307619328. Current Snapshot: MvccSnapshot[applied={T|T < 7317718985524355072}]
I20260812 16:34:46.730549 2517 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:34:46.730674 2517 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:34:46.731619 2517 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7640a127d2f44227b38cd312c486a133: Sending DeleteTablet for 3 replicas of tablet aa8cafd0ff4540f5a8e0fc16cc2748e0
I20260812 16:34:46.732050 2791 tablet_service.cc:1514] Processing DeleteTablet for tablet aa8cafd0ff4540f5a8e0fc16cc2748e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:34:46 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:37828
I20260812 16:34:46.732296 2946 tablet_service.cc:1514] Processing DeleteTablet for tablet aa8cafd0ff4540f5a8e0fc16cc2748e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:34:46 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:43500
I20260812 16:34:46.732353 2648 tablet_service.cc:1514] Processing DeleteTablet for tablet aa8cafd0ff4540f5a8e0fc16cc2748e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:34:46 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:45254
I20260812 16:34:46.732589 3078 tablet_replica.cc:330] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79: stopping tablet replica
I20260812 16:34:46.733042 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 2557
I20260812 16:34:46.736675 3078 raft_consensus.cc:2229] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:34:46.736887 3078 raft_consensus.cc:2258] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:34:46.739650 3077 tablet_replica.cc:330] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3: stopping tablet replica
I20260812 16:34:46.739918 3078 ts_tablet_manager.cc:1905] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:34:46.742044 3078 ts_tablet_manager.cc:1918] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:34:46.742197 3078 log.cc:1196] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/aa8cafd0ff4540f5a8e0fc16cc2748e0
I20260812 16:34:46.742588 3078 ts_tablet_manager.cc:1939] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P c1126bc5d2e84fc1be4b8012ae372a79: Deleting consensus metadata
I20260812 16:34:46.743259 2498 catalog_manager.cc:4841] TS c1126bc5d2e84fc1be4b8012ae372a79 (127.19.238.67:45927): tablet aa8cafd0ff4540f5a8e0fc16cc2748e0 (table test-table [id=15ee4ddfeb434ee383985d1716a2cb59]) successfully deleted
I20260812 16:34:46.744414 3077 raft_consensus.cc:2229] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:34:46.744589 3077 raft_consensus.cc:2258] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:34:46.745184 3077 ts_tablet_manager.cc:1905] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 16:34:46.748044 2498 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.238.65:39681 (error 108)
I20260812 16:34:46.749987 3077 ts_tablet_manager.cc:1918] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:34:46.750178 3077 log.cc:1196] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/aa8cafd0ff4540f5a8e0fc16cc2748e0
I20260812 16:34:46.750530 3077 ts_tablet_manager.cc:1939] T aa8cafd0ff4540f5a8e0fc16cc2748e0 P 3e9381d6c94f42d38b0c9d4d5ef058a3: Deleting consensus metadata
I20260812 16:34:46.751281 2500 catalog_manager.cc:4841] TS 3e9381d6c94f42d38b0c9d4d5ef058a3 (127.19.238.66:43033): tablet aa8cafd0ff4540f5a8e0fc16cc2748e0 (table test-table [id=15ee4ddfeb434ee383985d1716a2cb59]) successfully deleted
I20260812 16:34:46.752279 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:46.752144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:39681 (local address 127.0.0.1:45268)
0812 16:34:46.752199 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:46.752242 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:39681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 16:34:46.752521 2498 catalog_manager.cc:4584] TS 5db58a364a034627a46072903439e7cb (127.19.238.65:39681): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aa8cafd0ff4540f5a8e0fc16cc2748e0: Network error: Client connection negotiation failed: client connection to 127.19.238.65:39681: connect: Connection refused (error 111)
I20260812 16:34:46.755988 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 2732
I20260812 16:34:46.766784 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 2877
I20260812 16:34:46.805209 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:46.805051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:39681 (local address 127.0.0.1:45270)
0812 16:34:46.805121 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:46.805156 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:39681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 16:34:46.805626 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:46.805544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:39681 (local address 127.0.0.1:45276)
0812 16:34:46.805580 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:46.805601 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:39681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 16:34:46.813302 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 2474
I20260812 16:34:46.822950 20409 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 16:34:47.168794 20409 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 16:34:47.168900 20409 mini_ranger.cc:66] Stopping Ranger...
I20260812 16:34:47.514232 20409 mini_ranger.cc:68] Stopped Ranger
2026-08-12 16:34:47.514 UTC [1450] LOG: received smart shutdown request
2026-08-12 16:34:47.519 UTC [1450] LOG: background worker "logical replication launcher" (PID 1459) exited with exit code 1
2026-08-12 16:34:47.520 UTC [1454] LOG: shutting down
2026-08-12 16:34:47.719 UTC [1450] LOG: database system is shut down
2026-08-12T16:34:47Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50737 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:34:47 dist-test-slave-mvvj krb5kdc[3090](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:34:47 dist-test-slave-mvvj krb5kdc[3090](info): set up 2 sockets
Aug 12 16:34:47 dist-test-slave-mvvj krb5kdc[3090](info): commencing operation
krb5kdc: starting...
W20260812 16:34:48.992115 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:34:49 dist-test-slave-mvvj krb5kdc[3090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:34:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:34:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:34:49.127215 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:45771
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:45771,127.19.238.125:43069,127.19.238.124:41065
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:34:49.211727 3104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:49.212067 3104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:49.212108 3104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:49.213276 3104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:34:49.213347 3104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:49.213374 3104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:49.213406 3104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:34:49.213438 3104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:34:49.214987 3104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:45771,127.19.238.125:43069,127.19.238.124:41065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:45771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:49.215296 3104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:49.215546 3104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:49.220809 3111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:49.220824 3110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:49.220813 3113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:49.221062 3104 server_base.cc:1034] running on GCE node
I20260812 16:34:49.221320 3104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:49.221649 3104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:49.222905 3104 hybrid_clock.cc:648] HybridClock initialized: now 1786552489222863 us; error 61 us; skew 500 ppm
Aug 12 16:34:49 dist-test-slave-mvvj krb5kdc[3090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552489, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:49.225793 3104 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:34:49.226279 3104 webserver.cc:460] Webserver started at http://127.19.238.126:41891/ using document root <none> and password file <none>
I20260812 16:34:49.226548 3104 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:49.226622 3104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:49.226780 3104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:34:49.228008 3104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc"
format_stamp: "Formatted at 2026-08-12 16:34:49 on dist-test-slave-mvvj"
server_key: "c682e4c11c2d70906f0e3958d8e3d046"
server_key_iv: "0fc41ca17df1465df902de165ed6be61"
server_key_version: "encryptionkey@0"
I20260812 16:34:49.228391 3104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc"
format_stamp: "Formatted at 2026-08-12 16:34:49 on dist-test-slave-mvvj"
server_key: "c682e4c11c2d70906f0e3958d8e3d046"
server_key_iv: "0fc41ca17df1465df902de165ed6be61"
server_key_version: "encryptionkey@0"
I20260812 16:34:49.229801 3104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:34:49.230683 3120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:49.230896 3104 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:34:49.231014 3104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc"
format_stamp: "Formatted at 2026-08-12 16:34:49 on dist-test-slave-mvvj"
server_key: "c682e4c11c2d70906f0e3958d8e3d046"
server_key_iv: "0fc41ca17df1465df902de165ed6be61"
server_key_version: "encryptionkey@0"
I20260812 16:34:49.231092 3104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:49.274346 3104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:49.276731 3104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:49.276937 3104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:49.281351 3104 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:45771
I20260812 16:34:49.281425 3174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:45771 every 8 connection(s)
I20260812 16:34:49.281821 3104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:34:49.282848 3175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:34:49.283761 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 3104
I20260812 16:34:49.283946 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:34:49.284183 20409 external_mini_cluster.cc:1428] Setting key eca8ceeb36075aba45241372f2c9fa6c
I20260812 16:34:49.285276 3175 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:34:49 dist-test-slave-mvvj krb5kdc[3090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552489, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.238.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.125@KRBTEST.COM" created.
I20260812 16:34:49.296456 3179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.286551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38888 (local address 127.19.238.126:45771)
0812 16:34:49.286707 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:49.286711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:49.286747 (+ 36us) server_negotiation.cc:407] Connection header received
0812 16:34:49.287297 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:49.287299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:49.287327 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:49.287441 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:49.288119 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.289051 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.289989 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.290323 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.290461 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.292846 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:49.292853 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:49.292854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:49.292861 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:49.294453 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.294890 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.294894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.294895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.294985 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.295225 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.295226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.295226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.295522 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:49.295615 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:49.295923 (+ 308us) server_negotiation.cc:299] Negotiation successful
0812 16:34:49.296234 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":152,"server-negotiator.queue_time_us":127,"thread_start_us":70,"threads_started":1}
I20260812 16:34:49.296466 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.286297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:45771 (local address 127.0.0.1:38888)
0812 16:34:49.286523 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.286563 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:49.287117 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:49.287471 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:49.287474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:49.287767 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:49.287996 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:49.288001 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.289178 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.289179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:49.289878 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:49.289880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.290435 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.290437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:49.290511 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.291148 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:49.291157 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:49.292729 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:49.294565 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:49.294572 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:49.294575 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:49.294787 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:49.295060 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:49.295061 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:49.295061 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:49.295156 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:49.295605 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:49.295611 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:49.295682 (+ 71us) client_negotiation.cc:769] Sending connection context
0812 16:34:49.295794 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 16:34:49.296106 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":118,"threads_started":1}
I20260812 16:34:49.297724 3175 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:34:49.298136 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.298000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:43069 (local address 127.0.0.1:45760)
0812 16:34:49.298055 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.298104 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.125:43069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 16:34:49.298286 3122 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.238.125:43069: connect: Connection refused (error 111)
I20260812 16:34:49.298750 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.298654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:43069 (local address 127.0.0.1:45770)
0812 16:34:49.298703 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.298728 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.125:43069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 16:34:49.298892 3175 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.125:43069: Network error: Client connection negotiation failed: client connection to 127.19.238.125:43069: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.125.keytab.
Entry for principal kudu/127.19.238.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.125.keytab.
I20260812 16:34:49.315133 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.125:43069
--webserver_interface=127.19.238.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.125.keytab
--principal=kudu/127.19.238.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:45771,127.19.238.125:43069,127.19.238.124:41065
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:34:49.348171 3175 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } attempt: 1
I20260812 16:34:49.348763 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.348606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:43069 (local address 127.0.0.1:45778)
0812 16:34:49.348677 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.348726 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.125:43069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 16:34:49.349156 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.349055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:43069 (local address 127.0.0.1:45788)
0812 16:34:49.349102 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.349132 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.125:43069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 16:34:49.349265 3175 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.125:43069: Network error: Client connection negotiation failed: client connection to 127.19.238.125:43069: connect: Connection refused (error 111)
W20260812 16:34:49.401237 3183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:49.401479 3183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:49.401504 3183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:49.402731 3183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:34:49.402782 3183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:49.402796 3183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:49.402809 3183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:34:49.402822 3183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:34:49.404511 3183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:45771,127.19.238.125:43069,127.19.238.124:41065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.125.keytab
--principal=kudu/127.19.238.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.125:43069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.238.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:49.404812 3183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:49.405076 3183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:49.410588 3190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:49.410598 3192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:49.410643 3183 server_base.cc:1034] running on GCE node
W20260812 16:34:49.410598 3189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:49.411078 3183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:49.411424 3183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:49.412595 3183 hybrid_clock.cc:648] HybridClock initialized: now 1786552489412580 us; error 43 us; skew 500 ppm
Aug 12 16:34:49 dist-test-slave-mvvj krb5kdc[3090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552489, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:49.415378 3183 init.cc:377] Logged in from keytab as kudu/127.19.238.125@KRBTEST.COM (short username kudu)
I20260812 16:34:49.415931 3183 webserver.cc:460] Webserver started at http://127.19.238.125:40765/ using document root <none> and password file <none>
I20260812 16:34:49.416158 3183 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:49.416203 3183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:49.416311 3183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:34:49.417423 3175 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } attempt: 2
I20260812 16:34:49.417490 3183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-1/data/instance:
uuid: "b217d5cf36b144788dac9951d2a9e84e"
format_stamp: "Formatted at 2026-08-12 16:34:49 on dist-test-slave-mvvj"
server_key: "0ba2200481ce31d124f93013b1d30b22"
server_key_iv: "fb55f948cec4e152a01accadd18ccb83"
server_key_version: "encryptionkey@0"
I20260812 16:34:49.417974 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.417793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:43069 (local address 127.0.0.1:45792)
0812 16:34:49.417888 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.417930 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.125:43069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 16:34:49.417971 3183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-1/wal/instance:
uuid: "b217d5cf36b144788dac9951d2a9e84e"
format_stamp: "Formatted at 2026-08-12 16:34:49 on dist-test-slave-mvvj"
server_key: "0ba2200481ce31d124f93013b1d30b22"
server_key_iv: "fb55f948cec4e152a01accadd18ccb83"
server_key_version: "encryptionkey@0"
I20260812 16:34:49.418438 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.418312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:43069 (local address 127.0.0.1:45794)
0812 16:34:49.418376 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.418409 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.125:43069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 16:34:49.418551 3175 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.125:43069: Network error: Client connection negotiation failed: client connection to 127.19.238.125:43069: connect: Connection refused (error 111)
I20260812 16:34:49.419584 3183 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:34:49.420495 3200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:49.420814 3183 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:34:49.420902 3183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-1/wal
uuid: "b217d5cf36b144788dac9951d2a9e84e"
format_stamp: "Formatted at 2026-08-12 16:34:49 on dist-test-slave-mvvj"
server_key: "0ba2200481ce31d124f93013b1d30b22"
server_key_iv: "fb55f948cec4e152a01accadd18ccb83"
server_key_version: "encryptionkey@0"
I20260812 16:34:49.421005 3183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:49.432216 3183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:49.434782 3183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:49.434970 3183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:49.439538 3183 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.125:43069
I20260812 16:34:49.439594 3256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.125:43069 every 8 connection(s)
I20260812 16:34:49.440013 3183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-1/data/info.pb
I20260812 16:34:49.441088 3257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:34:49.441465 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 3183
I20260812 16:34:49.441594 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-1/wal/instance
I20260812 16:34:49.441797 20409 external_mini_cluster.cc:1428] Setting key 21880a2eabe41bfb0ed31a399bf92108
I20260812 16:34:49.442965 3257 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:34:49 dist-test-slave-mvvj krb5kdc[3090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552489, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.125@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.238.124@KRBTEST.COM; defaulting to no policy
I20260812 16:34:49.453353 3179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.443797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38898 (local address 127.19.238.126:45771)
0812 16:34:49.443912 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:49.443915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:49.444150 (+ 235us) server_negotiation.cc:407] Connection header received
0812 16:34:49.444899 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:49.444900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:49.444913 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:49.444964 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:49.445838 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.446336 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.447411 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.447637 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.447719 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.450237 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:49.450238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:49.450239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:49.450249 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:49.451760 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.452269 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.452271 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.452271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.452333 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.452606 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.452607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.452607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.452730 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:49.452809 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:49.453119 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 16:34:49.453211 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 16:34:49.453485 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.443745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:45771 (local address 127.0.0.1:38898)
0812 16:34:49.443978 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.444034 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:49.444749 (+ 715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:49.445035 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:49.445039 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:49.445450 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:49.445742 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:49.445749 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.446456 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.446458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:49.447298 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:49.447301 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.447731 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.447733 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:49.447923 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.448539 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:49.448547 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:49.450133 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:49.451869 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:49.451877 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:49.451880 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:49.452178 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:49.452402 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:49.452403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:49.452403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:49.452530 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:49.452800 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:49.452807 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:49.452962 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 16:34:49.453072 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 16:34:49.453245 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":138,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.124@KRBTEST.COM" created.
I20260812 16:34:49.454130 3257 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:34:49 dist-test-slave-mvvj krb5kdc[3090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552489, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.125@KRBTEST.COM for kudu/127.19.238.125@KRBTEST.COM
I20260812 16:34:49.464178 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.454413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:43069 (local address 127.0.0.1:45808)
0812 16:34:49.454466 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.454478 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:49.454521 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:49.454824 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:49.454825 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:49.454945 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:49.454996 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:49.454997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.456324 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.456325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:49.457382 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:49.457386 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.458213 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.458215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:49.458323 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.458949 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:49.458955 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:49.460883 (+ 1928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:49.462983 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:49.462984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:49.462984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:49.463155 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:49.463390 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:49.463391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:49.463391 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:49.463448 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:49.463775 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:49.463776 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:49.463814 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 16:34:49.463910 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 16:34:49.464033 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 16:34:49.464932 3262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.454498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45808 (local address 127.19.238.125:43069)
0812 16:34:49.454654 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:49.454659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:49.454686 (+ 27us) server_negotiation.cc:407] Connection header received
0812 16:34:49.454709 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:49.454711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:49.454725 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:49.454803 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:49.455077 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.456197 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.457495 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.458098 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.458197 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.460977 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:49.460989 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:49.460989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:49.460996 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:49.462898 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.463254 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.463258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.463258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.463321 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.463524 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.463525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.463526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.463696 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:49.463787 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:49.464728 (+ 941us) server_negotiation.cc:299] Negotiation successful
0812 16:34:49.464814 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260812 16:34:49.465695 3257 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:34:49.466419 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.465942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:41065 (local address 127.0.0.1:48260)
0812 16:34:49.466346 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.466397 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376}
W20260812 16:34:49.466571 3204 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.238.124:41065: connect: Connection refused (error 111)
I20260812 16:34:49.467044 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.466949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:41065 (local address 127.0.0.1:48270)
0812 16:34:49.466996 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.467021 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 16:34:49.467159 3257 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.124:41065: Network error: Client connection negotiation failed: client connection to 127.19.238.124:41065: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.124.keytab.
Entry for principal kudu/127.19.238.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.124.keytab.
I20260812 16:34:49.480331 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.124:41065
--webserver_interface=127.19.238.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.124.keytab
--principal=kudu/127.19.238.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:45771,127.19.238.125:43069,127.19.238.124:41065
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:34:49.509717 3175 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } attempt: 3
Aug 12 16:34:49 dist-test-slave-mvvj krb5kdc[3090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552489, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for kudu/127.19.238.125@KRBTEST.COM
I20260812 16:34:49.516409 3257 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } attempt: 1
I20260812 16:34:49.516858 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.516704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:41065 (local address 127.0.0.1:48274)
0812 16:34:49.516775 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.516816 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 16:34:49.517622 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.517496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:41065 (local address 127.0.0.1:48278)
0812 16:34:49.517552 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.517585 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 16:34:49.517947 3262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.510269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45812 (local address 127.19.238.125:43069)
0812 16:34:49.510342 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:49.510345 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:49.510358 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:34:49.510428 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:49.510429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:49.510442 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:49.510534 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:49.510917 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.511530 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.512370 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.512610 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.512703 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.514795 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:49.514796 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:49.514797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:49.514813 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:49.516693 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.517122 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.517123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.517123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.517181 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.517410 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.517410 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.517411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.517559 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:49.517634 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:49.517819 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 16:34:49.517891 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 16:34:49.517977 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.510196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:43069 (local address 127.0.0.1:45812)
0812 16:34:49.510272 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.510283 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:49.510354 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:49.510538 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:49.510539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:49.510736 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:49.510810 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:49.510813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.511635 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.511637 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:49.512268 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:49.512271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.512701 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.512702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:49.512793 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.513324 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:49.513329 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:49.514682 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:49.516807 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:49.516808 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:49.516809 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:49.517044 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:49.517283 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:49.517284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:49.517285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:49.517353 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:49.517636 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:49.517638 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:49.517699 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 16:34:49.517770 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 16:34:49.517881 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 16:34:49.518100 3257 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.124:41065: Network error: Client connection negotiation failed: client connection to 127.19.238.124:41065: connect: Connection refused (error 111)
I20260812 16:34:49.518627 3175 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:34:49.519063 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.518930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:41065 (local address 127.0.0.1:48280)
0812 16:34:49.518986 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.519025 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 16:34:49.519465 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.519358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:41065 (local address 127.0.0.1:48290)
0812 16:34:49.519411 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.519442 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 16:34:49.519595 3175 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.124:41065: Network error: Client connection negotiation failed: client connection to 127.19.238.124:41065: connect: Connection refused (error 111)
I20260812 16:34:49.550781 3175 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } attempt: 1
I20260812 16:34:49.551447 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.551265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:41065 (local address 127.0.0.1:48296)
0812 16:34:49.551351 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.551410 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 16:34:49.551915 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.551801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:41065 (local address 127.0.0.1:48302)
0812 16:34:49.551869 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.551896 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 16:34:49.552059 3175 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.124:41065: Network error: Client connection negotiation failed: client connection to 127.19.238.124:41065: connect: Connection refused (error 111)
W20260812 16:34:49.568321 3265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:49.568547 3265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:49.568569 3265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:49.569732 3265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:34:49.569775 3265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:49.569789 3265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:49.569802 3265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:34:49.569814 3265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:34:49.571331 3265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:45771,127.19.238.125:43069,127.19.238.124:41065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.124.keytab
--principal=kudu/127.19.238.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.124:41065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.238.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:49.571604 3265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:49.571892 3265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:49.576959 3270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:49.576952 3273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:49.576952 3271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:49.577463 3265 server_base.cc:1034] running on GCE node
I20260812 16:34:49.577761 3265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:49.578128 3265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:49.579339 3265 hybrid_clock.cc:648] HybridClock initialized: now 1786552489579313 us; error 50 us; skew 500 ppm
Aug 12 16:34:49 dist-test-slave-mvvj krb5kdc[3090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552489, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:49.582198 3265 init.cc:377] Logged in from keytab as kudu/127.19.238.124@KRBTEST.COM (short username kudu)
I20260812 16:34:49.582754 3265 webserver.cc:460] Webserver started at http://127.19.238.124:35971/ using document root <none> and password file <none>
I20260812 16:34:49.583011 3265 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:49.583082 3265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:49.583238 3265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:34:49.584461 3265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-2/data/instance:
uuid: "d925c114fad0467c8dc5e062e4c00cfa"
format_stamp: "Formatted at 2026-08-12 16:34:49 on dist-test-slave-mvvj"
server_key: "efc151cb7d5aaaf8921fbe79e5c54d43"
server_key_iv: "fbf295809af3fb04a1a6c5458704b6fb"
server_key_version: "encryptionkey@0"
I20260812 16:34:49.584841 3265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-2/wal/instance:
uuid: "d925c114fad0467c8dc5e062e4c00cfa"
format_stamp: "Formatted at 2026-08-12 16:34:49 on dist-test-slave-mvvj"
server_key: "efc151cb7d5aaaf8921fbe79e5c54d43"
server_key_iv: "fbf295809af3fb04a1a6c5458704b6fb"
server_key_version: "encryptionkey@0"
I20260812 16:34:49.586239 3257 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } attempt: 2
I20260812 16:34:49.586319 3265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:34:49.586714 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.586578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:41065 (local address 127.0.0.1:48314)
0812 16:34:49.586642 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.586680 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 16:34:49.587204 3280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:49.587220 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.587045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:41065 (local address 127.0.0.1:48318)
0812 16:34:49.587145 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.587187 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260812 16:34:49.587423 3257 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.124:41065: Network error: Client connection negotiation failed: client connection to 127.19.238.124:41065: connect: Connection refused (error 111)
I20260812 16:34:49.587548 3265 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:34:49.587664 3265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-2/wal
uuid: "d925c114fad0467c8dc5e062e4c00cfa"
format_stamp: "Formatted at 2026-08-12 16:34:49 on dist-test-slave-mvvj"
server_key: "efc151cb7d5aaaf8921fbe79e5c54d43"
server_key_iv: "fbf295809af3fb04a1a6c5458704b6fb"
server_key_version: "encryptionkey@0"
I20260812 16:34:49.587760 3265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:49.614140 3265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:49.616778 3265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:49.616961 3265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:49.621464 3265 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.124:41065
I20260812 16:34:49.621516 3335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.124:41065 every 8 connection(s)
I20260812 16:34:49.621934 3265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-2/data/info.pb
I20260812 16:34:49.622872 3336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:34:49.624708 3336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:34:49.626780 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 3265
I20260812 16:34:49.626927 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786552327177731-20409-0/minicluster-data/master-2/wal/instance
I20260812 16:34:49.627175 20409 external_mini_cluster.cc:1428] Setting key c5eb7be1577080d2b8359453cfef6769
I20260812 16:34:49.627236 3175 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } attempt: 2
Aug 12 16:34:49 dist-test-slave-mvvj krb5kdc[3090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552489, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.124@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
Aug 12 16:34:49 dist-test-slave-mvvj krb5kdc[3090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552489, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for kudu/127.19.238.124@KRBTEST.COM
Aug 12 16:34:49 dist-test-slave-mvvj krb5kdc[3090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.124@KRBTEST.COM
I20260812 16:34:49.636433 3179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.625591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38908 (local address 127.19.238.126:45771)
0812 16:34:49.625691 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:49.625693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:49.625871 (+ 178us) server_negotiation.cc:407] Connection header received
0812 16:34:49.626404 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:49.626406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:49.626419 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:49.626461 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:49.627575 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.628385 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.629625 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.629898 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.629967 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.632725 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:49.632726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:49.632727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:49.632732 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:49.634581 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.635135 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.635136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.635137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.635200 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.635501 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.635502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.635502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.635741 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:49.635818 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:49.636244 (+ 426us) server_negotiation.cc:299] Negotiation successful
0812 16:34:49.636340 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 16:34:49.636667 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.625476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:45771 (local address 127.0.0.1:38908)
0812 16:34:49.625730 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.625782 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:49.626282 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:49.626599 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:49.626604 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:49.627079 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:49.627448 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:49.627456 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.628470 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.628471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:49.629531 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:49.629534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.629985 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.629986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:49.630146 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.630813 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:49.630824 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:49.632628 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:49.634663 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:49.634669 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:49.634671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:49.635014 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:49.635276 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:49.635277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:49.635278 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:49.635422 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:49.635824 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:49.635854 (+ 30us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:49.636035 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 16:34:49.636219 (+ 184us) client_negotiation.cc:240] Negotiation successful
0812 16:34:49.636394 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":93,"threads_started":1}
I20260812 16:34:49.636861 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.627596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:41065 (local address 127.0.0.1:48338)
0812 16:34:49.627659 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.627688 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:49.627730 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:49.628352 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:49.628353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:49.628561 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:49.628593 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:49.628594 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.629798 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.629799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:49.630347 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:49.630349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.630990 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.630991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:49.631053 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.631482 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:49.631489 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:49.633078 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:49.635633 (+ 2555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:49.635634 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:49.635634 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:49.635906 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:49.636161 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:49.636162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:49.636162 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:49.636223 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:49.636578 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:49.636579 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:49.636622 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 16:34:49.636669 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 16:34:49.636758 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 16:34:49.637369 3341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.627964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48338 (local address 127.19.238.124:41065)
0812 16:34:49.628186 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:49.628188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:49.628204 (+ 16us) server_negotiation.cc:407] Connection header received
0812 16:34:49.628226 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:49.628227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:49.628244 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:49.628651 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:49.628678 (+ 27us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.629719 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.630453 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.630906 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.630987 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.633181 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:49.633189 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:49.633190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:49.633197 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:49.635304 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.635999 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.636003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.636003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.636084 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.636295 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.636296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.636297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.636468 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:49.637114 (+ 646us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:49.637172 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 16:34:49.637264 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":126,"threads_started":1}
I20260812 16:34:49.637640 3336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:34:49.638290 3342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.627953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48324 (local address 127.19.238.124:41065)
0812 16:34:49.628168 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:49.628173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:49.628197 (+ 24us) server_negotiation.cc:407] Connection header received
0812 16:34:49.628217 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:49.628218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:49.628241 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:49.628334 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:49.629087 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.629765 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.630817 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.631112 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.631628 (+ 516us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.634266 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:49.634267 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:49.634268 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:49.634275 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:49.636274 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.636670 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.636671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.636671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.636755 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.636958 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.636960 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.636960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.637089 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:49.637635 (+ 546us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:49.638064 (+ 429us) server_negotiation.cc:299] Negotiation successful
0812 16:34:49.638146 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":152,"threads_started":1}
I20260812 16:34:49.639304 3175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc: Bootstrap starting.
I20260812 16:34:49.640039 3175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc: Neither blocks nor log segments found. Creating new log.
I20260812 16:34:49.640390 3175 log.cc:824] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc: Log is configured to *not* fsync() on all Append() calls
I20260812 16:34:49.641498 3175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc: No bootstrap required, opened a new log
Aug 12 16:34:49 dist-test-slave-mvvj krb5kdc[3090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552489, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.124@KRBTEST.COM for kudu/127.19.238.125@KRBTEST.COM
I20260812 16:34:49.644575 3175 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } } peers { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } } peers { permanent_uuid: "d925c114fad0467c8dc5e062e4c00cfa" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } }
I20260812 16:34:49.644745 3175 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:34:49.644793 3175 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf91f5ed5c6c4c5aa54f047bfc88f5bc, State: Initialized, Role: FOLLOWER
I20260812 16:34:49.644953 3175 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } } peers { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } } peers { permanent_uuid: "d925c114fad0467c8dc5e062e4c00cfa" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } }
I20260812 16:34:49.645076 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.637928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:43069 (local address 127.0.0.1:45814)
0812 16:34:49.637979 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.637989 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:49.638027 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:49.638152 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:49.638153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:49.638291 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:49.638375 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:49.638378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.639040 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.639041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:49.639594 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:49.639595 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.640049 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.640050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:49.640128 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.640572 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:49.640577 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:49.641985 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:49.643910 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:49.643911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:49.643911 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:49.644116 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:49.644331 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:49.644332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:49.644332 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:49.644389 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:49.644707 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:49.644708 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:49.644754 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 16:34:49.644852 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 16:34:49.644965 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 16:34:49.645092 3262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.637993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45814 (local address 127.19.238.125:43069)
0812 16:34:49.638042 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:49.638044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:49.638052 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:34:49.638079 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:49.638079 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:49.638091 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:49.638152 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:49.638486 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.638954 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.639678 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.639965 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.640051 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.642088 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:49.642089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:49.642090 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:49.642097 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:49.643782 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.644203 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.644204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.644205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.644258 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.644461 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.644462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.644462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.644609 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:49.644656 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:49.644898 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 16:34:49.644985 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 16:34:49.645426 3175 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:34:49.645474 3336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:34:49.646184 3344 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } } peers { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } } peers { permanent_uuid: "d925c114fad0467c8dc5e062e4c00cfa" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } } }
I20260812 16:34:49.646404 3344 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc [sys.catalog]: This master's current role is: FOLLOWER
Aug 12 16:34:49 dist-test-slave-mvvj krb5kdc[3090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
W20260812 16:34:49.647517 3355 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:34:49.647615 3355 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 12 16:34:49 dist-test-slave-mvvj krb5kdc[3090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552489, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.124@KRBTEST.COM for kudu/127.19.238.124@KRBTEST.COM
I20260812 16:34:49.650656 3179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.641681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38912 (local address 127.19.238.126:45771)
0812 16:34:49.641718 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:49.641720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:49.641731 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:34:49.641820 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:49.641820 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:49.641830 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:49.641859 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:49.642517 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.643198 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.644212 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.644494 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.644537 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.647672 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:49.647674 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:49.647674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:49.647680 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:49.649358 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.649768 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.649769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.649770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.649825 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.650038 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.650038 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.650039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.650165 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:49.650266 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:49.650487 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 16:34:49.650574 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:34:49.648370 3104 master_runner.cc:186] Error getting master registration for 127.19.238.125:43069: OK, instance_id { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" instance_seqno: 1786552489438279 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 16:34:49.651013 3104 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 16:34:49.653525 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.645708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:41065 (local address 127.0.0.1:48344)
0812 16:34:49.645810 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.645819 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:49.645849 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:49.646079 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:49.646080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:49.646234 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:49.646271 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:49.646274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.647141 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.647141 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:49.647903 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:49.647907 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.648420 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.648421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:49.648466 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.648904 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:49.648910 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:49.650418 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:49.652311 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:49.652312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:49.652313 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:49.652506 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:49.652727 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:49.652728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:49.652728 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:49.652796 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:49.653164 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:49.653165 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:49.653205 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 16:34:49.653288 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 16:34:49.653398 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260812 16:34:49.653534 3342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.645766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48344 (local address 127.19.238.124:41065)
0812 16:34:49.645954 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:49.645956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:49.645964 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:34:49.645985 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:49.645986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:49.645997 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:49.646074 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:49.646352 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.647052 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.647995 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.648304 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.648375 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.650504 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:49.650506 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:49.650506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:49.650513 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:49.652211 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.652586 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.652588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.652588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.652649 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.652864 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.652865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.652865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.653077 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:49.653154 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:49.653331 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 16:34:49.653451 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":166}
I20260812 16:34:49.655120 3336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa: Bootstrap starting.
I20260812 16:34:49.655781 3336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa: Neither blocks nor log segments found. Creating new log.
I20260812 16:34:49.656126 3336 log.cc:824] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa: Log is configured to *not* fsync() on all Append() calls
Aug 12 16:34:49 dist-test-slave-mvvj krb5kdc[3090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.125@KRBTEST.COM
I20260812 16:34:49.656955 3336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa: No bootstrap required, opened a new log
I20260812 16:34:49.658727 3336 raft_consensus.cc:348] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } } peers { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } } peers { permanent_uuid: "d925c114fad0467c8dc5e062e4c00cfa" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } }
I20260812 16:34:49.658882 3336 raft_consensus.cc:374] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:34:49.658928 3336 raft_consensus.cc:729] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d925c114fad0467c8dc5e062e4c00cfa, State: Initialized, Role: FOLLOWER
I20260812 16:34:49.659080 3336 consensus_queue.cc:260] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } } peers { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } } peers { permanent_uuid: "d925c114fad0467c8dc5e062e4c00cfa" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } }
I20260812 16:34:49.659372 3358 sys_catalog.cc:455] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } } peers { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } } peers { permanent_uuid: "d925c114fad0467c8dc5e062e4c00cfa" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } } }
I20260812 16:34:49.659438 3336 sys_catalog.cc:564] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:34:49.659521 3358 sys_catalog.cc:458] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:34:49.660185 3262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.651576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45826 (local address 127.19.238.125:43069)
0812 16:34:49.651614 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:49.651616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:49.651624 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:34:49.651672 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:49.651673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:49.651683 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:49.651739 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:49.652294 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.652917 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.653994 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.654278 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.654352 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.657071 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:49.657073 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:49.657073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:49.657081 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:49.658776 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.659183 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.659185 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.659185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.659242 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.659446 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.659446 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.659447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.659584 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:49.659727 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:49.660007 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 16:34:49.660089 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:34:49.661804 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 16:34:49.661959 3369 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:34:49.662031 3369 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:34:49.664657 3265 master_runner.cc:186] Error getting master registration for 127.19.238.125:43069: OK, instance_id { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" instance_seqno: 1786552489438279 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 16:34:49.666999 3265 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 16:34:49.672400 3262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.663383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45828 (local address 127.19.238.125:43069)
0812 16:34:49.663473 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:49.663475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:49.663981 (+ 506us) server_negotiation.cc:407] Connection header received
0812 16:34:49.664096 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:49.664098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:49.664110 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:49.664165 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:49.664921 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.665673 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.666916 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.667213 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.667266 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.669145 (+ 1879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:49.669146 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:49.669147 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:49.669154 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:49.671001 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.671430 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.671431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.671431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.671484 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.671702 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.671703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.671703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.671905 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:49.671985 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:49.672214 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 16:34:49.672305 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 16:34:49.672550 3179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.663430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38922 (local address 127.19.238.126:45771)
0812 16:34:49.663528 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:49.663530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:49.664190 (+ 660us) server_negotiation.cc:407] Connection header received
0812 16:34:49.664283 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:49.664284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:49.664296 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:49.664328 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:49.664987 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.665684 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.666917 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.667211 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.667255 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.669121 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:49.669123 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:49.669123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:49.669132 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:49.671279 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.671667 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.671668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.671668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.671722 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.671960 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.671961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.671961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.672094 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:49.672166 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:49.672360 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 16:34:49.672437 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 16:34:49.678575 3257 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } attempt: 3
I20260812 16:34:49.679425 3342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.663869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48356 (local address 127.19.238.124:41065)
0812 16:34:49.664054 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:49.664056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:49.673139 (+ 9083us) server_negotiation.cc:407] Connection header received
0812 16:34:49.673334 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:49.673336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:49.673349 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:49.673381 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:49.673981 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.674468 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.675145 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.675349 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.675437 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.676634 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:49.676636 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:49.676636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:49.676645 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:49.678060 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.678451 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.678452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.678453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.678558 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.678778 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.678779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.678780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.678918 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:49.678990 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:49.679229 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 16:34:49.679323 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
Aug 12 16:34:49 dist-test-slave-mvvj krb5kdc[3090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552489, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.125@KRBTEST.COM for kudu/127.19.238.124@KRBTEST.COM
I20260812 16:34:49.686486 3341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.679190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48364 (local address 127.19.238.124:41065)
0812 16:34:49.679231 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:49.679233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:49.679240 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:34:49.679258 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:49.679259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:49.679267 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:49.679315 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:49.679891 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.680564 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.681237 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.681469 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.681553 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.683327 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:49.683329 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:49.683329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:49.683335 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:49.684930 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.685671 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.685672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.685673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.685728 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:49.685941 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:49.685942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:49.685942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:49.686067 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:49.686141 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:49.686303 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 16:34:49.686384 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:34:49.686486 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:49.678891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:41065 (local address 127.0.0.1:48364)
0812 16:34:49.679037 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:49.679049 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:49.679111 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:49.679355 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:49.679356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:49.679514 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:49.679561 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:49.679563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.680647 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.680648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:49.681139 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:49.681141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:49.681552 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:49.681553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:49.681613 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:49.682047 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:49.682052 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:49.683234 (+ 1182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:49.685297 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:49.685299 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:49.685299 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:49.685587 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:49.685805 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:49.685806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:49.685806 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:49.685874 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:49.686145 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:49.686146 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:49.686192 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 16:34:49.686278 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 16:34:49.686385 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260812 16:34:49.688349 3257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e: Bootstrap starting.
I20260812 16:34:49.689190 3257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e: Neither blocks nor log segments found. Creating new log.
I20260812 16:34:49.689558 3257 log.cc:824] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:34:49.696575 3257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e: No bootstrap required, opened a new log
I20260812 16:34:49.698325 3257 raft_consensus.cc:348] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } } peers { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } } peers { permanent_uuid: "d925c114fad0467c8dc5e062e4c00cfa" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } }
I20260812 16:34:49.698496 3257 raft_consensus.cc:374] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:34:49.698549 3257 raft_consensus.cc:729] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b217d5cf36b144788dac9951d2a9e84e, State: Initialized, Role: FOLLOWER
I20260812 16:34:49.698683 3257 consensus_queue.cc:260] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } } peers { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } } peers { permanent_uuid: "d925c114fad0467c8dc5e062e4c00cfa" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } }
I20260812 16:34:49.699038 3379 sys_catalog.cc:455] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } } peers { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } } peers { permanent_uuid: "d925c114fad0467c8dc5e062e4c00cfa" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } } }
I20260812 16:34:49.699102 3257 sys_catalog.cc:564] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:34:49.699216 3379 sys_catalog.cc:458] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e [sys.catalog]: This master's current role is: FOLLOWER
W20260812 16:34:49.701237 3391 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:34:49.701359 3391 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:34:49.703692 3183 master_runner.cc:263] No leader master found from master b217d5cf36b144788dac9951d2a9e84e
I20260812 16:34:49.703748 3183 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 16:34:49.777899 3126 tablet.cc:2369] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:34:49.867682 3287 tablet.cc:2369] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:34:49.886475 3358 raft_consensus.cc:482] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:34:49.886667 3358 raft_consensus.cc:504] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } } peers { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } } peers { permanent_uuid: "d925c114fad0467c8dc5e062e4c00cfa" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } }
I20260812 16:34:49.886976 3358 leader_election.cc:288] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf91f5ed5c6c4c5aa54f047bfc88f5bc (127.19.238.126:45771), b217d5cf36b144788dac9951d2a9e84e (127.19.238.125:43069)
I20260812 16:34:49.887270 3147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d925c114fad0467c8dc5e062e4c00cfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" is_pre_election: true
I20260812 16:34:49.887410 3147 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d925c114fad0467c8dc5e062e4c00cfa in term 0.
I20260812 16:34:49.887418 3228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d925c114fad0467c8dc5e062e4c00cfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b217d5cf36b144788dac9951d2a9e84e" is_pre_election: true
I20260812 16:34:49.887629 3228 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d925c114fad0467c8dc5e062e4c00cfa in term 0.
I20260812 16:34:49.887676 3284 leader_election.cc:302] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf91f5ed5c6c4c5aa54f047bfc88f5bc, d925c114fad0467c8dc5e062e4c00cfa; no voters:
I20260812 16:34:49.887854 3358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:34:49.887931 3358 raft_consensus.cc:482] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:34:49.887949 3358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:34:49.888727 3358 raft_consensus.cc:504] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } } peers { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } } peers { permanent_uuid: "d925c114fad0467c8dc5e062e4c00cfa" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } }
I20260812 16:34:49.888871 3358 leader_election.cc:288] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [CANDIDATE]: Term 1 election: Requested vote from peers bf91f5ed5c6c4c5aa54f047bfc88f5bc (127.19.238.126:45771), b217d5cf36b144788dac9951d2a9e84e (127.19.238.125:43069)
I20260812 16:34:49.889039 3228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d925c114fad0467c8dc5e062e4c00cfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b217d5cf36b144788dac9951d2a9e84e"
I20260812 16:34:49.889063 3147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d925c114fad0467c8dc5e062e4c00cfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc"
I20260812 16:34:49.889143 3228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:34:49.889168 3147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:34:49.889892 3147 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d925c114fad0467c8dc5e062e4c00cfa in term 1.
I20260812 16:34:49.889974 3228 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d925c114fad0467c8dc5e062e4c00cfa in term 1.
I20260812 16:34:49.890097 3284 leader_election.cc:302] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf91f5ed5c6c4c5aa54f047bfc88f5bc, d925c114fad0467c8dc5e062e4c00cfa; no voters:
I20260812 16:34:49.890205 3358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:34:49.890437 3358 raft_consensus.cc:686] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [term 1 LEADER]: Becoming Leader. State: Replica: d925c114fad0467c8dc5e062e4c00cfa, State: Running, Role: LEADER
I20260812 16:34:49.890532 3358 consensus_queue.cc:237] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } } peers { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } } peers { permanent_uuid: "d925c114fad0467c8dc5e062e4c00cfa" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } }
I20260812 16:34:49.891016 3393 sys_catalog.cc:455] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader d925c114fad0467c8dc5e062e4c00cfa. Latest consensus state: current_term: 1 leader_uuid: "d925c114fad0467c8dc5e062e4c00cfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } } peers { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } } peers { permanent_uuid: "d925c114fad0467c8dc5e062e4c00cfa" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } } }
I20260812 16:34:49.891119 3393 sys_catalog.cc:458] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [sys.catalog]: This master's current role is: LEADER
I20260812 16:34:49.891371 3395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:34:49.891582 3395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:34:49.893114 3228 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e [term 1 FOLLOWER]: Refusing update from remote peer d925c114fad0467c8dc5e062e4c00cfa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:34:49.893126 3147 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc [term 1 FOLLOWER]: Refusing update from remote peer d925c114fad0467c8dc5e062e4c00cfa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:34:49.893460 3393 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:34:49.893603 3358 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:34:49.894941 3344 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader d925c114fad0467c8dc5e062e4c00cfa. Latest consensus state: current_term: 1 leader_uuid: "d925c114fad0467c8dc5e062e4c00cfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } } peers { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } } peers { permanent_uuid: "d925c114fad0467c8dc5e062e4c00cfa" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } } }
I20260812 16:34:49.894951 3379 sys_catalog.cc:455] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d925c114fad0467c8dc5e062e4c00cfa. Latest consensus state: current_term: 1 leader_uuid: "d925c114fad0467c8dc5e062e4c00cfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } } peers { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } } peers { permanent_uuid: "d925c114fad0467c8dc5e062e4c00cfa" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } } }
I20260812 16:34:49.895045 3344 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:34:49.895066 3379 sys_catalog.cc:458] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:34:49.896136 3393 sys_catalog.cc:455] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d925c114fad0467c8dc5e062e4c00cfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } } peers { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } } peers { permanent_uuid: "d925c114fad0467c8dc5e062e4c00cfa" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } } }
I20260812 16:34:49.896230 3393 sys_catalog.cc:458] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [sys.catalog]: This master's current role is: LEADER
I20260812 16:34:49.896278 3379 sys_catalog.cc:455] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d925c114fad0467c8dc5e062e4c00cfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } } peers { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } } peers { permanent_uuid: "d925c114fad0467c8dc5e062e4c00cfa" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } } }
I20260812 16:34:49.896373 3379 sys_catalog.cc:458] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:34:49.896492 3395 catalog_manager.cc:1347] Generated new cluster ID: 9a01257654464234a29ff711ad7e4567
I20260812 16:34:49.896539 3395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:34:49.896716 3396 mvcc.cc:205] Tried to move back new op lower bound from 7317718998599331840 to 7317718998592393216. Current Snapshot: MvccSnapshot[applied={T|T < 7317718998599331840 or (T in {7317718998599331840})}]
I20260812 16:34:49.896792 3358 sys_catalog.cc:455] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d925c114fad0467c8dc5e062e4c00cfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } } peers { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } } peers { permanent_uuid: "d925c114fad0467c8dc5e062e4c00cfa" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } } }
I20260812 16:34:49.896860 3358 sys_catalog.cc:458] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa [sys.catalog]: This master's current role is: LEADER
I20260812 16:34:49.897440 3344 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d925c114fad0467c8dc5e062e4c00cfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf91f5ed5c6c4c5aa54f047bfc88f5bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 45771 } } peers { permanent_uuid: "b217d5cf36b144788dac9951d2a9e84e" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 43069 } } peers { permanent_uuid: "d925c114fad0467c8dc5e062e4c00cfa" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 41065 } } }
I20260812 16:34:49.897528 3344 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:34:49.904167 3395 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:34:49.904803 3395 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:34:49.910660 3395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d925c114fad0467c8dc5e062e4c00cfa: Generated new TSK 0
I20260812 16:34:49.910943 3395 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:34:50.648275 3355 catalog_manager.cc:1259] Loaded cluster ID: 9a01257654464234a29ff711ad7e4567
I20260812 16:34:50.648365 3355 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc: loading cluster ID for follower catalog manager: success
I20260812 16:34:50.649425 3355 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc: acquiring CA information for follower catalog manager: success
I20260812 16:34:50.649796 3355 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bf91f5ed5c6c4c5aa54f047bfc88f5bc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:34:50.701901 3391 catalog_manager.cc:1259] Loaded cluster ID: 9a01257654464234a29ff711ad7e4567
I20260812 16:34:50.701988 3391 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e: loading cluster ID for follower catalog manager: success
I20260812 16:34:50.702798 3391 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e: acquiring CA information for follower catalog manager: success
I20260812 16:34:50.703078 3391 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b217d5cf36b144788dac9951d2a9e84e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:34:50.970403 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 3104
I20260812 16:34:50.978515 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 3183
I20260812 16:34:50.985755 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 3265
2026-08-12T16:34:50Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3138 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:34:51 dist-test-slave-mvvj krb5kdc[3414](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:34:51 dist-test-slave-mvvj krb5kdc[3414](info): set up 2 sockets
Aug 12 16:34:51 dist-test-slave-mvvj krb5kdc[3414](info): commencing operation
krb5kdc: starting...
W20260812 16:34:52.124933 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:34:52 dist-test-slave-mvvj krb5kdc[3414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:34:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:34:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:34:52.264434 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:41879
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:41295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:41879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:34:52.350715 3427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:52.350952 3427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:52.350978 3427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:52.352238 3427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:34:52.352283 3427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:52.352298 3427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:52.352317 3427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:34:52.352330 3427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:34:52.353828 3427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:41295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:41879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:41879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:52.354049 3427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:52.354228 3427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:52.359393 3432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:52.359385 3433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:52.359390 3435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:52.359725 3427 server_base.cc:1034] running on GCE node
I20260812 16:34:52.360065 3427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:52.360457 3427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:52.361636 3427 hybrid_clock.cc:648] HybridClock initialized: now 1786552492361621 us; error 32 us; skew 500 ppm
Aug 12 16:34:52 dist-test-slave-mvvj krb5kdc[3414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552492, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:52.364552 3427 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:34:52.365093 3427 webserver.cc:460] Webserver started at http://127.19.238.126:33363/ using document root <none> and password file <none>
I20260812 16:34:52.365347 3427 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:52.365397 3427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:52.365548 3427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:34:52.366667 3427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "9f0a46427f5c47a8b72b5f38a5474521"
format_stamp: "Formatted at 2026-08-12 16:34:52 on dist-test-slave-mvvj"
server_key: "95e9bb3bc02d101ebea803bc0c43e19f"
server_key_iv: "523613a82ad1f3027c3b61ba2f6b3aa0"
server_key_version: "encryptionkey@0"
I20260812 16:34:52.367074 3427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "9f0a46427f5c47a8b72b5f38a5474521"
format_stamp: "Formatted at 2026-08-12 16:34:52 on dist-test-slave-mvvj"
server_key: "95e9bb3bc02d101ebea803bc0c43e19f"
server_key_iv: "523613a82ad1f3027c3b61ba2f6b3aa0"
server_key_version: "encryptionkey@0"
I20260812 16:34:52.368674 3427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:34:52.369779 3443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:52.369984 3427 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:34:52.370108 3427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "9f0a46427f5c47a8b72b5f38a5474521"
format_stamp: "Formatted at 2026-08-12 16:34:52 on dist-test-slave-mvvj"
server_key: "95e9bb3bc02d101ebea803bc0c43e19f"
server_key_iv: "523613a82ad1f3027c3b61ba2f6b3aa0"
server_key_version: "encryptionkey@0"
I20260812 16:34:52.370196 3427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:52.388471 3427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:52.390931 3427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:52.391121 3427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:52.395903 3427 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:41879
I20260812 16:34:52.395954 3499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:41879 every 8 connection(s)
I20260812 16:34:52.396373 3427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:34:52.397380 3500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:34:52.400205 3500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521: Bootstrap starting.
I20260812 16:34:52.400772 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 3427
I20260812 16:34:52.400909 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:34:52.400877 3500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521: Neither blocks nor log segments found. Creating new log.
I20260812 16:34:52.401126 20409 external_mini_cluster.cc:1428] Setting key bfc39111ea073a34948229962669cbb5
I20260812 16:34:52.401290 3500 log.cc:824] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521: Log is configured to *not* fsync() on all Append() calls
I20260812 16:34:52.402253 3500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521: No bootstrap required, opened a new log
I20260812 16:34:52.404775 3500 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0a46427f5c47a8b72b5f38a5474521" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 41879 } }
I20260812 16:34:52.404975 3500 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:34:52.405030 3500 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f0a46427f5c47a8b72b5f38a5474521, State: Initialized, Role: FOLLOWER
I20260812 16:34:52.405175 3500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0a46427f5c47a8b72b5f38a5474521" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 41879 } }
I20260812 16:34:52.405260 3500 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:34:52.405284 3500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:34:52.405359 3500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:34:52.406029 3500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0a46427f5c47a8b72b5f38a5474521" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 41879 } }
I20260812 16:34:52.406129 3500 leader_election.cc:302] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f0a46427f5c47a8b72b5f38a5474521; no voters:
I20260812 16:34:52.406370 3500 leader_election.cc:288] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:34:52.406430 3505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:34:52.406605 3505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521 [term 1 LEADER]: Becoming Leader. State: Replica: 9f0a46427f5c47a8b72b5f38a5474521, State: Running, Role: LEADER
I20260812 16:34:52.406688 3500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:34:52.406802 3505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0a46427f5c47a8b72b5f38a5474521" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 41879 } }
I20260812 16:34:52.407239 3505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f0a46427f5c47a8b72b5f38a5474521. Latest consensus state: current_term: 1 leader_uuid: "9f0a46427f5c47a8b72b5f38a5474521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0a46427f5c47a8b72b5f38a5474521" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 41879 } } }
I20260812 16:34:52.407262 3506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f0a46427f5c47a8b72b5f38a5474521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0a46427f5c47a8b72b5f38a5474521" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 41879 } } }
I20260812 16:34:52.407409 3505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521 [sys.catalog]: This master's current role is: LEADER
I20260812 16:34:52.408378 3506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521 [sys.catalog]: This master's current role is: LEADER
W20260812 16:34:52.408983 3523 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:34:52.409191 3523 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:34:52.409298 3518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 16:34:52 dist-test-slave-mvvj krb5kdc[3414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:34:52.409497 3518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:34:52.411422 3518 catalog_manager.cc:1347] Generated new cluster ID: e901a901bc3940058de137df46caae7f
I20260812 16:34:52.411502 3518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:34:52.414088 3504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:52.401751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45016 (local address 127.19.238.126:41879)
0812 16:34:52.402079 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:52.402087 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:52.402116 (+ 29us) server_negotiation.cc:407] Connection header received
0812 16:34:52.402689 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:52.402698 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:52.402782 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:52.402963 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:52.403873 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:52.404882 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:52.405928 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:52.406603 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:52.406770 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:52.409940 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:52.409945 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:52.409945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:52.409967 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:52.412225 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:52.412663 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:52.412667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:52.412667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:52.412796 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:52.413018 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:52.413019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:52.413019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:52.413388 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:52.413571 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:52.413669 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 16:34:52.413835 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":125,"threads_started":1}
I20260812 16:34:52.415160 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:34:52.417960 3518 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:34:52.418810 3518 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:34:52.431500 3518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f0a46427f5c47a8b72b5f38a5474521: Generated new TSK 0
I20260812 16:34:52.431773 3518 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:34:52.469802 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 3427
2026-08-12T16:34:52Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1482 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:34:52 dist-test-slave-mvvj krb5kdc[3529](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:34:52 dist-test-slave-mvvj krb5kdc[3529](info): set up 2 sockets
Aug 12 16:34:52 dist-test-slave-mvvj krb5kdc[3529](info): commencing operation
krb5kdc: starting...
W20260812 16:34:53.608768 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:34:53 dist-test-slave-mvvj krb5kdc[3529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:34:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:34:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:34:53.744168 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:46389
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:43677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:46389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:34:53.832213 3543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:53.832506 3543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:53.832549 3543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:53.833822 3543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:34:53.833905 3543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:53.833931 3543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:53.833966 3543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:34:53.834015 3543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:34:53.835723 3543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:43677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:46389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:46389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:53.836066 3543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:53.836326 3543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:53.841629 3548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:53.841588 3551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:53.841596 3549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:53.841704 3543 server_base.cc:1034] running on GCE node
I20260812 16:34:53.842198 3543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:53.842502 3543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:53.843667 3543 hybrid_clock.cc:648] HybridClock initialized: now 1786552493843652 us; error 29 us; skew 500 ppm
Aug 12 16:34:53 dist-test-slave-mvvj krb5kdc[3529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552493, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:53.850257 3543 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:34:53.850858 3543 webserver.cc:460] Webserver started at http://127.19.238.126:40093/ using document root <none> and password file <none>
I20260812 16:34:53.851161 3543 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:53.851240 3543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:53.851429 3543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:34:53.852779 3543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "3ed0d01bd8c145b199d22029262a0a9d"
format_stamp: "Formatted at 2026-08-12 16:34:53 on dist-test-slave-mvvj"
server_key: "11596787d5c9a28766ca2c0a0d761e90"
server_key_iv: "c89cadbb89e66b4389516812d884a817"
server_key_version: "encryptionkey@0"
I20260812 16:34:53.853200 3543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "3ed0d01bd8c145b199d22029262a0a9d"
format_stamp: "Formatted at 2026-08-12 16:34:53 on dist-test-slave-mvvj"
server_key: "11596787d5c9a28766ca2c0a0d761e90"
server_key_iv: "c89cadbb89e66b4389516812d884a817"
server_key_version: "encryptionkey@0"
I20260812 16:34:53.854805 3543 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:34:53.855765 3558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:53.855991 3543 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:34:53.856109 3543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "3ed0d01bd8c145b199d22029262a0a9d"
format_stamp: "Formatted at 2026-08-12 16:34:53 on dist-test-slave-mvvj"
server_key: "11596787d5c9a28766ca2c0a0d761e90"
server_key_iv: "c89cadbb89e66b4389516812d884a817"
server_key_version: "encryptionkey@0"
I20260812 16:34:53.856191 3543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:53.869184 3543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:53.871692 3543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:53.871980 3543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:53.876888 3543 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:46389
I20260812 16:34:53.876919 3613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:46389 every 8 connection(s)
I20260812 16:34:53.877359 3543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:34:53.878562 3614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:34:53.880797 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 3543
I20260812 16:34:53.880959 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:34:53.881175 20409 external_mini_cluster.cc:1428] Setting key 3b734dadffe388ad4ce00620275c34ba
I20260812 16:34:53.881420 3614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d: Bootstrap starting.
I20260812 16:34:53.882292 3614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d: Neither blocks nor log segments found. Creating new log.
I20260812 16:34:53.882675 3614 log.cc:824] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:34:53.883616 3614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d: No bootstrap required, opened a new log
I20260812 16:34:53.886466 3614 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed0d01bd8c145b199d22029262a0a9d" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46389 } }
I20260812 16:34:53.886703 3614 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:34:53.886741 3614 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed0d01bd8c145b199d22029262a0a9d, State: Initialized, Role: FOLLOWER
I20260812 16:34:53.886896 3614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed0d01bd8c145b199d22029262a0a9d" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46389 } }
I20260812 16:34:53.886998 3614 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:34:53.887036 3614 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:34:53.887091 3614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:34:53.888170 3614 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed0d01bd8c145b199d22029262a0a9d" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46389 } }
I20260812 16:34:53.888320 3614 leader_election.cc:302] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ed0d01bd8c145b199d22029262a0a9d; no voters:
I20260812 16:34:53.888552 3614 leader_election.cc:288] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:34:53.888731 3621 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:34:53.888938 3614 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:34:53.888943 3621 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 1 LEADER]: Becoming Leader. State: Replica: 3ed0d01bd8c145b199d22029262a0a9d, State: Running, Role: LEADER
I20260812 16:34:53.889175 3621 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed0d01bd8c145b199d22029262a0a9d" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46389 } }
I20260812 16:34:53.889873 3622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ed0d01bd8c145b199d22029262a0a9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed0d01bd8c145b199d22029262a0a9d" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46389 } } }
I20260812 16:34:53.889973 3622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [sys.catalog]: This master's current role is: LEADER
Aug 12 16:34:53 dist-test-slave-mvvj krb5kdc[3529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:34:53.890236 3631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:34:53.890671 3624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ed0d01bd8c145b199d22029262a0a9d. Latest consensus state: current_term: 1 leader_uuid: "3ed0d01bd8c145b199d22029262a0a9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed0d01bd8c145b199d22029262a0a9d" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46389 } } }
I20260812 16:34:53.890759 3624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [sys.catalog]: This master's current role is: LEADER
I20260812 16:34:53.891891 3631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:34:53.894977 3631 catalog_manager.cc:1347] Generated new cluster ID: d8f016f9ce7040478b698fb59f67b4b3
I20260812 16:34:53.895056 3631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:34:53.895640 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:53.881788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35766 (local address 127.19.238.126:46389)
0812 16:34:53.882034 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:53.882044 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:53.882084 (+ 40us) server_negotiation.cc:407] Connection header received
0812 16:34:53.882818 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:53.882829 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:53.882910 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:53.883072 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:53.883906 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:53.885523 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:53.886424 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:53.887128 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:53.887260 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:53.890919 (+ 3659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:53.890925 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:53.890926 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:53.890952 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:53.893455 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:53.893896 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:53.893900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:53.893900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:53.894058 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:53.894308 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:53.894309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:53.894310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:53.894774 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:53.894988 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:53.895243 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 16:34:53.895400 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":112,"threads_started":1}
I20260812 16:34:53.932726 3631 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:34:53.933467 3631 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:34:53.938040 3631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d: Generated new TSK 0
I20260812 16:34:53.938287 3631 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
I20260812 16:34:53.980104 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:46389
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:43677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:34:54.086030 3643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:54.086320 3643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:54.086366 3643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:54.087610 3643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:54.087692 3643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:54.087736 3643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:34:54.089617 3643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:43677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:46389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:54.089972 3643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:54.090283 3643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:54.096073 3651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:54.096091 3653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:54.096233 3643 server_base.cc:1034] running on GCE node
W20260812 16:34:54.096094 3650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:54.096626 3643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:54.096916 3643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:54.098106 3643 hybrid_clock.cc:648] HybridClock initialized: now 1786552494098078 us; error 43 us; skew 500 ppm
Aug 12 16:34:54 dist-test-slave-mvvj krb5kdc[3529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552494, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:54.101183 3643 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:34:54.101763 3643 webserver.cc:460] Webserver started at http://127.19.238.65:43867/ using document root <none> and password file <none>
I20260812 16:34:54.102031 3643 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:54.102105 3643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:54.102260 3643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:34:54.103440 3643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "37ca7c03644b497d8cdc2fcaa310218b"
format_stamp: "Formatted at 2026-08-12 16:34:54 on dist-test-slave-mvvj"
server_key: "b596875b76f09b0e856d64c56f30755b"
server_key_iv: "9249a202c14803f39814718d2fde2569"
server_key_version: "encryptionkey@0"
I20260812 16:34:54.103932 3643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "37ca7c03644b497d8cdc2fcaa310218b"
format_stamp: "Formatted at 2026-08-12 16:34:54 on dist-test-slave-mvvj"
server_key: "b596875b76f09b0e856d64c56f30755b"
server_key_iv: "9249a202c14803f39814718d2fde2569"
server_key_version: "encryptionkey@0"
I20260812 16:34:54.105454 3643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:34:54.106364 3660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:54.106575 3643 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:34:54.106699 3643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "37ca7c03644b497d8cdc2fcaa310218b"
format_stamp: "Formatted at 2026-08-12 16:34:54 on dist-test-slave-mvvj"
server_key: "b596875b76f09b0e856d64c56f30755b"
server_key_iv: "9249a202c14803f39814718d2fde2569"
server_key_version: "encryptionkey@0"
I20260812 16:34:54.106789 3643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:54.126399 3643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:54.129045 3643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:54.129237 3643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:54.129519 3643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:34:54.129874 3643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:34:54.129931 3643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:54.129966 3643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:34:54.130012 3643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:54.136559 3643 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:45577
I20260812 16:34:54.136626 3787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:45577 every 8 connection(s)
I20260812 16:34:54.136981 3643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:34:54.137873 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 3643
I20260812 16:34:54.138019 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:34:54.138279 20409 external_mini_cluster.cc:1428] Setting key 9fbcad715cdab124af474eef451a5f71
Aug 12 16:34:54 dist-test-slave-mvvj krb5kdc[3529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552494, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.238.66@KRBTEST.COM; defaulting to no policy
I20260812 16:34:54.148415 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.137861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:48845 (local address 127.19.238.126:46389)
0812 16:34:54.137943 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:54.137946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:54.138237 (+ 291us) server_negotiation.cc:407] Connection header received
0812 16:34:54.138927 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:54.138928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:54.138939 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:54.139015 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:54.140247 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.141033 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.142077 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.142325 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.142528 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.145269 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:54.145271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:54.145272 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:54.145280 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:54.146799 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:54.147269 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:54.147270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:54.147271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:54.147338 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:54.147619 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:54.147620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:54.147621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:54.147793 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:54.147894 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:54.148206 (+ 312us) server_negotiation.cc:299] Negotiation successful
0812 16:34:54.148314 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 16:34:54.148638 3790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.137810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:48845)
0812 16:34:54.138088 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:54.138137 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:54.138810 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:54.139130 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:54.139134 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:54.139818 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:54.140128 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:54.140134 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.141130 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.141131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:54.141973 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:54.141975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.142541 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.142542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:54.142696 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.143352 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:54.143365 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:54.145158 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:54.146912 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:54.146922 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:54.146924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:54.147184 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:54.147419 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:54.147420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:54.147421 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:54.147550 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:54.147897 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:54.147906 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:54.148065 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 16:34:54.148186 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 16:34:54.148362 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":104,"threads_started":1}
I20260812 16:34:54.149312 3788 heartbeater.cc:347] Connected to a master server at 127.19.238.126:46389
I20260812 16:34:54.149437 3788 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:54.149642 3788 heartbeater.cc:511] Master 127.19.238.126:46389 requested a full tablet report, sending...
I20260812 16:34:54.150251 3576 ts_manager.cc:194] Registered new tserver with Master: 37ca7c03644b497d8cdc2fcaa310218b (127.19.238.65:45577)
I20260812 16:34:54.150959 3576 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:48845
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
I20260812 16:34:54.171219 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:46389
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:43677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:34:54.257512 3793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:54.257753 3793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:54.257781 3793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:54.259107 3793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:54.259173 3793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:54.259204 3793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:34:54.260987 3793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:43677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:46389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:54.261291 3793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:54.261647 3793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:54.267230 3801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:54.267293 3793 server_base.cc:1034] running on GCE node
W20260812 16:34:54.267249 3798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:54.267254 3799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:54.267653 3793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:54.267985 3793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:54.269182 3793 hybrid_clock.cc:648] HybridClock initialized: now 1786552494269172 us; error 38 us; skew 500 ppm
Aug 12 16:34:54 dist-test-slave-mvvj krb5kdc[3529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552494, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:54.271963 3793 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:34:54.272403 3793 webserver.cc:460] Webserver started at http://127.19.238.66:37955/ using document root <none> and password file <none>
I20260812 16:34:54.272606 3793 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:54.272648 3793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:54.272746 3793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:34:54.273808 3793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "1466609a58564979844b45740b1b65ca"
format_stamp: "Formatted at 2026-08-12 16:34:54 on dist-test-slave-mvvj"
server_key: "9d5c66a6216a18dd8d5b9db094744ab7"
server_key_iv: "64e8343c7c1f9ca05ea6e82914dd5ea9"
server_key_version: "encryptionkey@0"
I20260812 16:34:54.274142 3793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "1466609a58564979844b45740b1b65ca"
format_stamp: "Formatted at 2026-08-12 16:34:54 on dist-test-slave-mvvj"
server_key: "9d5c66a6216a18dd8d5b9db094744ab7"
server_key_iv: "64e8343c7c1f9ca05ea6e82914dd5ea9"
server_key_version: "encryptionkey@0"
I20260812 16:34:54.275591 3793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:34:54.276474 3809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:54.276656 3793 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:34:54.276770 3793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "1466609a58564979844b45740b1b65ca"
format_stamp: "Formatted at 2026-08-12 16:34:54 on dist-test-slave-mvvj"
server_key: "9d5c66a6216a18dd8d5b9db094744ab7"
server_key_iv: "64e8343c7c1f9ca05ea6e82914dd5ea9"
server_key_version: "encryptionkey@0"
I20260812 16:34:54.276855 3793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:54.292246 3793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:54.294451 3793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:54.294584 3793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:54.294806 3793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:34:54.295178 3793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:34:54.295214 3793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:54.295239 3793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:34:54.295277 3793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:54.301725 3793 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:41157
I20260812 16:34:54.301779 3936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:41157 every 8 connection(s)
I20260812 16:34:54.302170 3793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:34:54.307636 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 3793
I20260812 16:34:54.307776 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:34:54.308051 20409 external_mini_cluster.cc:1428] Setting key b7764c8c0b4032f7a771b79abe5e609d
Aug 12 16:34:54 dist-test-slave-mvvj krb5kdc[3529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552494, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:34:54.314370 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.303151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:43985 (local address 127.19.238.126:46389)
0812 16:34:54.303227 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:54.303229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:54.303435 (+ 206us) server_negotiation.cc:407] Connection header received
0812 16:34:54.303997 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:54.303998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:54.304014 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:54.304089 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:54.305084 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.305668 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.306841 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.307036 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.307099 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.310457 (+ 3358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:54.310458 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:54.310459 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:54.310468 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:54.312592 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:54.313112 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:54.313114 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:54.313114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:54.313190 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:54.313506 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:54.313507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:54.313507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:54.313686 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:54.313764 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:54.314106 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 16:34:54.314222 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 16:34:54.314448 3939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.303105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:43985)
0812 16:34:54.303269 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:54.303314 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:54.303879 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:54.304148 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:54.304152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:54.304748 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:54.304968 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:54.304973 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.305811 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.305812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:54.306735 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:54.306738 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.307131 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.307132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:54.307254 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.308057 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:54.308069 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:54.310339 (+ 2270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:54.312708 (+ 2369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:54.312719 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:54.312721 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:54.313018 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:54.313292 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:54.313293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:54.313293 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:54.313427 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:54.313761 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:54.313767 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:54.313946 (+ 179us) client_negotiation.cc:769] Sending connection context
0812 16:34:54.314043 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 16:34:54.314203 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260812 16:34:54.315204 3937 heartbeater.cc:347] Connected to a master server at 127.19.238.126:46389
I20260812 16:34:54.315308 3937 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:54.315557 3937 heartbeater.cc:511] Master 127.19.238.126:46389 requested a full tablet report, sending...
I20260812 16:34:54.316071 3576 ts_manager.cc:194] Registered new tserver with Master: 1466609a58564979844b45740b1b65ca (127.19.238.66:41157)
I20260812 16:34:54.316725 3576 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:43985
WARNING: no policy specified for kudu/127.19.238.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
I20260812 16:34:54.340286 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:46389
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:43677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:34:54.427899 3942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:54.428150 3942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:54.428180 3942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:54.429435 3942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:54.429484 3942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:54.429514 3942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:34:54.431121 3942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:43677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:46389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:54.431417 3942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:54.431677 3942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:54.437157 3951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:54.437157 3948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:54.437249 3947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:54.437389 3942 server_base.cc:1034] running on GCE node
I20260812 16:34:54.437662 3942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:54.437986 3942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:54.439157 3942 hybrid_clock.cc:648] HybridClock initialized: now 1786552494439132 us; error 43 us; skew 500 ppm
Aug 12 16:34:54 dist-test-slave-mvvj krb5kdc[3529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552494, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:54.442004 3942 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:34:54.442648 3942 webserver.cc:460] Webserver started at http://127.19.238.67:40213/ using document root <none> and password file <none>
I20260812 16:34:54.442937 3942 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:54.443020 3942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:54.443193 3942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:34:54.444516 3942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7"
format_stamp: "Formatted at 2026-08-12 16:34:54 on dist-test-slave-mvvj"
server_key: "79fb17f6ba27d64d1793bf63467ee5dc"
server_key_iv: "146a360e71ce92f32dcefe5eac702919"
server_key_version: "encryptionkey@0"
I20260812 16:34:54.444939 3942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7"
format_stamp: "Formatted at 2026-08-12 16:34:54 on dist-test-slave-mvvj"
server_key: "79fb17f6ba27d64d1793bf63467ee5dc"
server_key_iv: "146a360e71ce92f32dcefe5eac702919"
server_key_version: "encryptionkey@0"
I20260812 16:34:54.446525 3942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 16:34:54.447403 3961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:54.447628 3942 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:34:54.447753 3942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7"
format_stamp: "Formatted at 2026-08-12 16:34:54 on dist-test-slave-mvvj"
server_key: "79fb17f6ba27d64d1793bf63467ee5dc"
server_key_iv: "146a360e71ce92f32dcefe5eac702919"
server_key_version: "encryptionkey@0"
I20260812 16:34:54.447881 3942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:54.465137 3942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:54.468266 3942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:54.468461 3942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:54.468743 3942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:34:54.469092 3942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:34:54.469149 3942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:54.469195 3942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:34:54.469221 3942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:54.478185 3942 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:37851
I20260812 16:34:54.478245 4091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:37851 every 8 connection(s)
I20260812 16:34:54.478755 3942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
Aug 12 16:34:54 dist-test-slave-mvvj krb5kdc[3529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552494, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:34:54.487231 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 3942
I20260812 16:34:54.487327 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:34:54.487527 20409 external_mini_cluster.cc:1428] Setting key 53d13ddc900dfc673db995496c54cff6
I20260812 16:34:54.491211 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.479562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:54915 (local address 127.19.238.126:46389)
0812 16:34:54.479641 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:54.479644 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:54.479968 (+ 324us) server_negotiation.cc:407] Connection header received
0812 16:34:54.480556 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:54.480558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:54.480571 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:54.480641 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:54.481715 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.482410 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.483516 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.483873 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.483958 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.486799 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:54.486800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:54.486801 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:54.486810 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:54.489388 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:54.489979 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:54.489980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:54.489980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:54.490042 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:54.490391 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:54.490392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:54.490393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:54.490577 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:54.490648 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:54.491061 (+ 413us) server_negotiation.cc:299] Negotiation successful
0812 16:34:54.491140 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 16:34:54.491442 4095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.479553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:54915)
0812 16:34:54.479759 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:54.479809 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:54.480434 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:54.480718 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:54.480722 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:54.481397 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:54.481604 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:54.481609 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.482546 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.482547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:54.483438 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:54.483440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.484007 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.484008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:54.484192 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.484873 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:54.484882 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:54.486681 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:54.489495 (+ 2814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:54.489504 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:54.489506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:54.489898 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:54.490148 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:54.490150 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:54.490150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:54.490285 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:54.490669 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:54.490676 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:54.490857 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 16:34:54.491007 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 16:34:54.491171 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
I20260812 16:34:54.492103 4092 heartbeater.cc:347] Connected to a master server at 127.19.238.126:46389
I20260812 16:34:54.492195 4092 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:54.492442 4092 heartbeater.cc:511] Master 127.19.238.126:46389 requested a full tablet report, sending...
I20260812 16:34:54.493005 3576 ts_manager.cc:194] Registered new tserver with Master: caa889dc6e294f9fa9b6df0fb0c1d1e7 (127.19.238.67:37851)
I20260812 16:34:54.493408 3576 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:54915
I20260812 16:34:54.500031 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:34:54.507448 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.501502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35774 (local address 127.19.238.126:46389)
0812 16:34:54.501575 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:54.501577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:54.501712 (+ 135us) server_negotiation.cc:407] Connection header received
0812 16:34:54.501760 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:54.501761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:54.501771 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:54.501827 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:54.502467 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.502957 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.503669 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.503915 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.503975 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.505158 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:54.505159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:54.505160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:54.505167 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:54.506418 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:54.506766 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:54.506766 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:54.506767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:54.506818 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:54.506985 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:54.506986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:54.506986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:54.507137 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:54.507219 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:54.507304 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 16:34:54.507366 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 16:34:54.509447 3576 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35774:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:34:54.511677 3576 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:34:54.519217 4107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.514733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:37851 (local address 127.0.0.1:32846)
0812 16:34:54.514971 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:54.515025 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:54.515104 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:54.515553 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:54.515557 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:54.515559 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:34:54.515672 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:54.515674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.516865 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.516866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:54.518008 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:54.518010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.518810 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.518811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:54.518937 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.518951 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 16:34:54.519047 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 16:34:54.519114 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":1,"thread_start_us":111,"threads_started":1}
I20260812 16:34:54.519625 4109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.515056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32846 (local address 127.19.238.67:37851)
0812 16:34:54.515272 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:54.515277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:54.515289 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:34:54.515342 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:54.515345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:54.515363 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:54.515449 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:34:54.515744 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.516741 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.518113 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.518686 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.518785 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.518859 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:54.519481 (+ 622us) server_negotiation.cc:299] Negotiation successful
0812 16:34:54.519533 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":88,"threads_started":1}
I20260812 16:34:54.520509 4014 tablet_service.cc:1467] Processing CreateTablet for tablet f81d78092616454493598626f23dc480 (DEFAULT_TABLE table=test-table [id=8a9b97e61d854d49a665267861dc4626]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:34:54.520669 4105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.514738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:45577 (local address 127.0.0.1:33510)
0812 16:34:54.515196 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:54.515209 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:54.515273 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:54.515943 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:54.515944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:54.515946 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:34:54.516011 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:54.516013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.517522 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.517523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:54.518938 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:54.518941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.520390 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.520391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:54.520466 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.520470 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:34:54.520547 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 16:34:54.520594 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":56,"thread_start_us":139,"threads_started":1}
I20260812 16:34:54.520695 4108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.514974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33510 (local address 127.19.238.65:45577)
0812 16:34:54.515531 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:54.515535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:54.515547 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:34:54.515592 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:54.515594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:54.515610 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:54.515696 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:34:54.516106 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.517384 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.519040 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.520280 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.520354 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.520395 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:54.520585 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 16:34:54.520629 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":99,"threads_started":1}
I20260812 16:34:54.520946 4014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f81d78092616454493598626f23dc480. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:34:54.521386 3715 tablet_service.cc:1467] Processing CreateTablet for tablet f81d78092616454493598626f23dc480 (DEFAULT_TABLE table=test-table [id=8a9b97e61d854d49a665267861dc4626]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:34:54.521636 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f81d78092616454493598626f23dc480. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:34:54.523986 4113 tablet_bootstrap.cc:492] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7: Bootstrap starting.
I20260812 16:34:54.524410 4106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.514733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:41157 (local address 127.0.0.1:55324)
0812 16:34:54.515235 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:54.515246 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:54.515291 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:54.516024 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:54.516025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:54.516028 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:34:54.516073 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:54.516074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.517522 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.517523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:54.518942 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:54.518944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.519956 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.519957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:54.524167 (+ 4210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.524176 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 16:34:54.524252 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 16:34:54.524311 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":90,"thread_start_us":73,"threads_started":1}
I20260812 16:34:54.524722 4113 tablet_bootstrap.cc:654] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7: Neither blocks nor log segments found. Creating new log.
I20260812 16:34:54.524753 4110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.515086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55324 (local address 127.19.238.66:41157)
0812 16:34:54.515787 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:54.515791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:54.515801 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:34:54.515900 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:54.515903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:54.515922 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:54.516007 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:34:54.516141 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.517388 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.519057 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.519823 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.519942 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.520013 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:54.524609 (+ 4596us) server_negotiation.cc:299] Negotiation successful
0812 16:34:54.524674 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":87,"threads_started":1}
I20260812 16:34:54.524830 4114 tablet_bootstrap.cc:492] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b: Bootstrap starting.
I20260812 16:34:54.525424 4113 log.cc:824] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7: Log is configured to *not* fsync() on all Append() calls
I20260812 16:34:54.525552 4114 tablet_bootstrap.cc:654] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b: Neither blocks nor log segments found. Creating new log.
I20260812 16:34:54.525785 3862 tablet_service.cc:1467] Processing CreateTablet for tablet f81d78092616454493598626f23dc480 (DEFAULT_TABLE table=test-table [id=8a9b97e61d854d49a665267861dc4626]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:34:54.526122 4114 log.cc:824] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b: Log is configured to *not* fsync() on all Append() calls
I20260812 16:34:54.526221 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f81d78092616454493598626f23dc480. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:34:54.526481 4113 tablet_bootstrap.cc:492] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7: No bootstrap required, opened a new log
I20260812 16:34:54.526561 4113 ts_tablet_manager.cc:1397] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 16:34:54.527132 4114 tablet_bootstrap.cc:492] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b: No bootstrap required, opened a new log
I20260812 16:34:54.527201 4114 ts_tablet_manager.cc:1397] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:34:54.528723 4113 raft_consensus.cc:348] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:54.528906 4113 raft_consensus.cc:374] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:34:54.528945 4113 raft_consensus.cc:729] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caa889dc6e294f9fa9b6df0fb0c1d1e7, State: Initialized, Role: FOLLOWER
I20260812 16:34:54.529109 4113 consensus_queue.cc:260] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:54.529217 4114 raft_consensus.cc:348] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:54.529397 4114 raft_consensus.cc:374] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:34:54.529417 4113 ts_tablet_manager.cc:1428] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:34:54.529441 4114 raft_consensus.cc:729] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37ca7c03644b497d8cdc2fcaa310218b, State: Initialized, Role: FOLLOWER
I20260812 16:34:54.529554 4114 consensus_queue.cc:260] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:54.529608 4092 heartbeater.cc:503] Master 127.19.238.126:46389 was elected leader, sending a full tablet report...
I20260812 16:34:54.529611 4117 tablet_bootstrap.cc:492] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca: Bootstrap starting.
I20260812 16:34:54.529860 4114 ts_tablet_manager.cc:1428] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:34:54.529865 3788 heartbeater.cc:503] Master 127.19.238.126:46389 was elected leader, sending a full tablet report...
I20260812 16:34:54.530504 4117 tablet_bootstrap.cc:654] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca: Neither blocks nor log segments found. Creating new log.
I20260812 16:34:54.531174 4117 log.cc:824] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca: Log is configured to *not* fsync() on all Append() calls
I20260812 16:34:54.532073 4117 tablet_bootstrap.cc:492] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca: No bootstrap required, opened a new log
I20260812 16:34:54.532135 4117 ts_tablet_manager.cc:1397] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:34:54.533753 4117 raft_consensus.cc:348] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:54.533902 4117 raft_consensus.cc:374] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:34:54.533927 4117 raft_consensus.cc:729] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1466609a58564979844b45740b1b65ca, State: Initialized, Role: FOLLOWER
I20260812 16:34:54.534073 4117 consensus_queue.cc:260] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:54.534389 3937 heartbeater.cc:503] Master 127.19.238.126:46389 was elected leader, sending a full tablet report...
I20260812 16:34:54.534406 4117 ts_tablet_manager.cc:1428] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260812 16:34:54.552860 3938 tablet.cc:2369] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:34:54.565691 4119 raft_consensus.cc:482] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:34:54.565840 4119 raft_consensus.cc:504] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:54.566232 4119 leader_election.cc:288] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1466609a58564979844b45740b1b65ca (127.19.238.66:41157), 37ca7c03644b497d8cdc2fcaa310218b (127.19.238.65:45577)
I20260812 16:34:54.570245 4110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.566502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:43821 (local address 127.19.238.66:41157)
0812 16:34:54.566550 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:54.566552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:54.566561 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:34:54.566620 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:54.566621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:54.566635 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:54.566700 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:34:54.566924 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.567679 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.569210 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.569799 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.569891 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.569900 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:54.570127 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 16:34:54.570167 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:34:54.570245 4108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.566595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:48531 (local address 127.19.238.65:45577)
0812 16:34:54.566648 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:54.566650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:54.566740 (+ 90us) server_negotiation.cc:407] Connection header received
0812 16:34:54.566776 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:54.566777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:54.566788 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:54.566838 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:34:54.567021 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.567742 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.569211 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.569825 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.569893 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.569900 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:54.570126 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 16:34:54.570167 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:34:54.570269 4095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.566405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:41157 (local address 127.19.238.67:43821)
0812 16:34:54.566478 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:54.566492 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:54.566546 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:54.566729 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:54.566731 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:54.566732 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:34:54.566813 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:54.566816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.567766 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.567767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:54.569094 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:54.569097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.569931 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.569932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:54.570032 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.570033 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 16:34:54.570104 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 16:34:54.570163 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":57}
I20260812 16:34:54.570281 4123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.566431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:45577 (local address 127.19.238.67:48531)
0812 16:34:54.566624 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:54.566635 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:54.566729 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:54.566870 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:54.566871 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:54.566873 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:34:54.566944 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:54.566947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.567821 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.567821 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:54.569094 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:54.569097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.569931 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.569932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:54.570027 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.570032 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:34:54.570104 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 16:34:54.570162 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":28,"thread_start_us":120,"threads_started":1}
I20260812 16:34:54.570730 3885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f81d78092616454493598626f23dc480" candidate_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1466609a58564979844b45740b1b65ca" is_pre_election: true
I20260812 16:34:54.570921 3885 raft_consensus.cc:2454] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate caa889dc6e294f9fa9b6df0fb0c1d1e7 in term 0.
I20260812 16:34:54.570899 3735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f81d78092616454493598626f23dc480" candidate_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ca7c03644b497d8cdc2fcaa310218b" is_pre_election: true
I20260812 16:34:54.571101 3735 raft_consensus.cc:2454] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate caa889dc6e294f9fa9b6df0fb0c1d1e7 in term 0.
I20260812 16:34:54.571192 3964 leader_election.cc:302] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1466609a58564979844b45740b1b65ca, caa889dc6e294f9fa9b6df0fb0c1d1e7; no voters:
I20260812 16:34:54.571358 4119 raft_consensus.cc:2781] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:34:54.571449 4119 raft_consensus.cc:482] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:34:54.571481 4119 raft_consensus.cc:3037] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:34:54.572297 4119 raft_consensus.cc:504] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:54.572460 4119 leader_election.cc:288] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [CANDIDATE]: Term 1 election: Requested vote from peers 1466609a58564979844b45740b1b65ca (127.19.238.66:41157), 37ca7c03644b497d8cdc2fcaa310218b (127.19.238.65:45577)
I20260812 16:34:54.572618 3735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f81d78092616454493598626f23dc480" candidate_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ca7c03644b497d8cdc2fcaa310218b"
I20260812 16:34:54.572706 3735 raft_consensus.cc:3037] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:34:54.572748 3885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f81d78092616454493598626f23dc480" candidate_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1466609a58564979844b45740b1b65ca"
I20260812 16:34:54.572844 3885 raft_consensus.cc:3037] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:34:54.573657 3735 raft_consensus.cc:2454] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caa889dc6e294f9fa9b6df0fb0c1d1e7 in term 1.
I20260812 16:34:54.573771 3885 raft_consensus.cc:2454] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caa889dc6e294f9fa9b6df0fb0c1d1e7 in term 1.
I20260812 16:34:54.573833 3962 leader_election.cc:302] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37ca7c03644b497d8cdc2fcaa310218b, caa889dc6e294f9fa9b6df0fb0c1d1e7; no voters:
I20260812 16:34:54.573922 4119 raft_consensus.cc:2781] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:34:54.574090 4119 raft_consensus.cc:686] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 1 LEADER]: Becoming Leader. State: Replica: caa889dc6e294f9fa9b6df0fb0c1d1e7, State: Running, Role: LEADER
I20260812 16:34:54.574204 4119 consensus_queue.cc:237] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:54.575112 3576 catalog_manager.cc:5495] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to caa889dc6e294f9fa9b6df0fb0c1d1e7 (127.19.238.67). New cstate: current_term: 1 leader_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } health_report { overall_health: UNKNOWN } } }
I20260812 16:34:54.594148 4109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.590898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32860 (local address 127.19.238.67:37851)
0812 16:34:54.590958 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:54.590961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:54.590970 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:34:54.591042 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:54.591043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:54.591110 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:54.591212 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:34:54.591413 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.592028 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.593251 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.593493 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.593563 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.593751 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:34:54.593863 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:34:54.593939 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:54.593997 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 16:34:54.594017 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:34:54.596570 3885 raft_consensus.cc:1261] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca [term 1 FOLLOWER]: Refusing update from remote peer caa889dc6e294f9fa9b6df0fb0c1d1e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:34:54.596633 3735 raft_consensus.cc:1261] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Refusing update from remote peer caa889dc6e294f9fa9b6df0fb0c1d1e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:34:54.597009 4124 consensus_queue.cc:1035] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:34:54.597182 4119 consensus_queue.cc:1035] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:34:54.600392 4128 mvcc.cc:205] Tried to move back new op lower bound from 7317719017864200192 to 7317719017776705536. Current Snapshot: MvccSnapshot[applied={T|T < 7317719017864200192}]
I20260812 16:34:54.600112 4127 mvcc.cc:205] Tried to move back new op lower bound from 7317719017864200192 to 7317719017776705536. Current Snapshot: MvccSnapshot[applied={T|T < 7317719017864200192}]
I20260812 16:34:54.602851 3575 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:34:54.602972 3575 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:34:54.603891 3575 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d: Sending DeleteTablet for 3 replicas of tablet f81d78092616454493598626f23dc480
I20260812 16:34:54.604279 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 3643
I20260812 16:34:54.604310 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet f81d78092616454493598626f23dc480 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:34:54 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:33510
I20260812 16:34:54.604332 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet f81d78092616454493598626f23dc480 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:34:54 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:55324
I20260812 16:34:54.606627 4014 tablet_service.cc:1514] Processing DeleteTablet for tablet f81d78092616454493598626f23dc480 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:34:54 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:32846
I20260812 16:34:54.606863 4138 tablet_replica.cc:330] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca: stopping tablet replica
I20260812 16:34:54.606976 4138 raft_consensus.cc:2229] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:34:54.607088 4138 raft_consensus.cc:2258] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:34:54.607600 4138 ts_tablet_manager.cc:1905] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:34:54.607975 4139 tablet_replica.cc:330] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7: stopping tablet replica
I20260812 16:34:54.608143 4139 raft_consensus.cc:2229] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:34:54.608331 4139 raft_consensus.cc:2258] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:34:54.609004 4139 ts_tablet_manager.cc:1905] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:34:54.609691 4138 ts_tablet_manager.cc:1918] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:34:54.609778 4138 log.cc:1196] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/f81d78092616454493598626f23dc480
I20260812 16:34:54.610061 4138 ts_tablet_manager.cc:1939] T f81d78092616454493598626f23dc480 P 1466609a58564979844b45740b1b65ca: Deleting consensus metadata
I20260812 16:34:54.610880 3561 catalog_manager.cc:4841] TS 1466609a58564979844b45740b1b65ca (127.19.238.66:41157): tablet f81d78092616454493598626f23dc480 (table test-table [id=8a9b97e61d854d49a665267861dc4626]) successfully deleted
I20260812 16:34:54.611116 4139 ts_tablet_manager.cc:1918] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:34:54.611258 4139 log.cc:1196] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/f81d78092616454493598626f23dc480
I20260812 16:34:54.611694 4139 ts_tablet_manager.cc:1939] T f81d78092616454493598626f23dc480 P caa889dc6e294f9fa9b6df0fb0c1d1e7: Deleting consensus metadata
I20260812 16:34:54.612243 3562 catalog_manager.cc:4841] TS caa889dc6e294f9fa9b6df0fb0c1d1e7 (127.19.238.67:37851): tablet f81d78092616454493598626f23dc480 (table test-table [id=8a9b97e61d854d49a665267861dc4626]) successfully deleted
W20260812 16:34:54.613456 3559 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.238.65:45577 (error 108)
I20260812 16:34:54.615096 4106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.614926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:45577 (local address 127.0.0.1:33526)
0812 16:34:54.615003 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:54.615058 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:45577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 16:34:54.615288 3559 catalog_manager.cc:4584] TS 37ca7c03644b497d8cdc2fcaa310218b (127.19.238.65:45577): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f81d78092616454493598626f23dc480: Network error: Client connection negotiation failed: client connection to 127.19.238.65:45577: connect: Connection refused (error 111)
I20260812 16:34:54.616082 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 3793
I20260812 16:34:54.629434 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 3942
I20260812 16:34:54.640121 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 3543
I20260812 16:34:54.651010 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:46389
--webserver_interface=127.19.238.126
--webserver_port=40093
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:43677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:46389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:34:54.763234 4141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:54.763489 4141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:54.763513 4141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:54.764739 4141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:34:54.764783 4141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:54.764797 4141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:54.764810 4141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:34:54.764822 4141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:34:54.766405 4141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:43677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:46389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:46389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=40093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:54.766656 4141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:54.766849 4141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:54.776646 4149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:54.776669 4148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:54.776701 4152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:54.776844 4141 server_base.cc:1034] running on GCE node
I20260812 16:34:54.777235 4141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:54.777644 4141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:54.778839 4141 hybrid_clock.cc:648] HybridClock initialized: now 1786552494778805 us; error 58 us; skew 500 ppm
Aug 12 16:34:54 dist-test-slave-mvvj krb5kdc[3529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552494, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:54.781764 4141 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:34:54.782330 4141 webserver.cc:460] Webserver started at http://127.19.238.126:40093/ using document root <none> and password file <none>
I20260812 16:34:54.782608 4141 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:54.782706 4141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:54.784351 4141 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:34:54.785157 4160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:54.785348 4141 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:34:54.785493 4141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "3ed0d01bd8c145b199d22029262a0a9d"
format_stamp: "Formatted at 2026-08-12 16:34:53 on dist-test-slave-mvvj"
server_key: "11596787d5c9a28766ca2c0a0d761e90"
server_key_iv: "c89cadbb89e66b4389516812d884a817"
server_key_version: "encryptionkey@0"
I20260812 16:34:54.785949 4141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:54.802724 4141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:54.805420 4141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:54.805611 4141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:54.810388 4141 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:46389
I20260812 16:34:54.810441 4219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:46389 every 8 connection(s)
I20260812 16:34:54.810906 4141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:34:54.812798 4221 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:34:54.813567 4221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d: Bootstrap starting.
I20260812 16:34:54.818785 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 4141
I20260812 16:34:54.819195 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:45577
--local_ip_for_outbound_sockets=127.19.238.65
--tserver_master_addrs=127.19.238.126:46389
--webserver_port=43867
--webserver_interface=127.19.238.65
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:43677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:34:54.823238 4221 log.cc:824] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:34:54.825687 4221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:34:54.826095 4221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d: Bootstrap complete.
I20260812 16:34:54.828688 4221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed0d01bd8c145b199d22029262a0a9d" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46389 } }
I20260812 16:34:54.828949 4221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed0d01bd8c145b199d22029262a0a9d, State: Initialized, Role: FOLLOWER
I20260812 16:34:54.829128 4221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed0d01bd8c145b199d22029262a0a9d" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46389 } }
I20260812 16:34:54.829203 4221 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:34:54.829275 4221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:34:54.829330 4221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 1 FOLLOWER]: Advancing to term 2
I20260812 16:34:54.830499 4221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed0d01bd8c145b199d22029262a0a9d" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46389 } }
I20260812 16:34:54.830600 4221 leader_election.cc:302] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ed0d01bd8c145b199d22029262a0a9d; no voters:
I20260812 16:34:54.830832 4221 leader_election.cc:288] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 16:34:54.830881 4225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 2 FOLLOWER]: Leader election won for term 2
I20260812 16:34:54.831012 4225 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 2 LEADER]: Becoming Leader. State: Replica: 3ed0d01bd8c145b199d22029262a0a9d, State: Running, Role: LEADER
I20260812 16:34:54.831118 4225 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed0d01bd8c145b199d22029262a0a9d" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46389 } }
I20260812 16:34:54.831161 4221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:34:54.831436 4225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ed0d01bd8c145b199d22029262a0a9d. Latest consensus state: current_term: 2 leader_uuid: "3ed0d01bd8c145b199d22029262a0a9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed0d01bd8c145b199d22029262a0a9d" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46389 } } }
I20260812 16:34:54.831522 4225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [sys.catalog]: This master's current role is: LEADER
I20260812 16:34:54.831441 4226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3ed0d01bd8c145b199d22029262a0a9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed0d01bd8c145b199d22029262a0a9d" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46389 } } }
I20260812 16:34:54.831710 4226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [sys.catalog]: This master's current role is: LEADER
I20260812 16:34:54.831745 4229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:34:54.832744 4229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:34:54.832875 4229 catalog_manager.cc:1259] Loaded cluster ID: d8f016f9ce7040478b698fb59f67b4b3
I20260812 16:34:54.833112 4229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:34:54.833392 4229 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 16:34:54.836270 4229 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 16:34:54.838547 4229 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:34:54.838830 4229 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d: Loaded TSK: 0
I20260812 16:34:54.839092 4229 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 16:34:54.924717 4223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:54.924947 4223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:54.924975 4223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:54.926210 4223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:54.926258 4223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:54.926286 4223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:34:54.927814 4223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:43677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:45577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=43867
--tserver_master_addrs=127.19.238.126:46389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:54.928140 4223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:54.928400 4223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 16:34:54.934566 4223 server_base.cc:1034] running on GCE node
W20260812 16:34:54.934566 4252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:54.934710 4251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:54.934928 4254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:54.935171 4223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:54.935459 4223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:54.936632 4223 hybrid_clock.cc:648] HybridClock initialized: now 1786552494936623 us; error 31 us; skew 500 ppm
Aug 12 16:34:54 dist-test-slave-mvvj krb5kdc[3529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552494, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:54.939458 4223 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:34:54.940096 4223 webserver.cc:460] Webserver started at http://127.19.238.65:43867/ using document root <none> and password file <none>
I20260812 16:34:54.940341 4223 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:54.940430 4223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:54.941972 4223 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:34:54.942881 4264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:54.943115 4223 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:34:54.943231 4223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "37ca7c03644b497d8cdc2fcaa310218b"
format_stamp: "Formatted at 2026-08-12 16:34:54 on dist-test-slave-mvvj"
server_key: "b596875b76f09b0e856d64c56f30755b"
server_key_iv: "9249a202c14803f39814718d2fde2569"
server_key_version: "encryptionkey@0"
I20260812 16:34:54.943629 4223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:54.964744 4223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:54.967281 4223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:54.967473 4223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:54.967741 4223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:34:54.968276 4272 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 16:34:54.970233 4223 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 16:34:54.970304 4223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 16:34:54.970366 4223 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 16:34:54.971076 4223 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 16:34:54.971133 4223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 16:34:54.971197 4272 tablet_bootstrap.cc:492] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b: Bootstrap starting.
I20260812 16:34:54.978789 4223 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:45577
I20260812 16:34:54.978958 4397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:45577 every 8 connection(s)
I20260812 16:34:54.979252 4223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:34:54.982779 4272 log.cc:824] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b: Log is configured to *not* fsync() on all Append() calls
I20260812 16:34:54.985968 4272 tablet_bootstrap.cc:492] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:34:54.986305 4272 tablet_bootstrap.cc:492] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b: Bootstrap complete.
I20260812 16:34:54.986594 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 4223
I20260812 16:34:54.986886 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:41157
--local_ip_for_outbound_sockets=127.19.238.66
--tserver_master_addrs=127.19.238.126:46389
--webserver_port=37955
--webserver_interface=127.19.238.66
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:43677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:34:54.988875 4272 ts_tablet_manager.cc:1397] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.011s
I20260812 16:34:54.990331 4272 raft_consensus.cc:348] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
Aug 12 16:34:54 dist-test-slave-mvvj krb5kdc[3529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552494, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:34:54.990633 4272 raft_consensus.cc:729] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37ca7c03644b497d8cdc2fcaa310218b, State: Initialized, Role: FOLLOWER
I20260812 16:34:54.990787 4272 consensus_queue.cc:260] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:54.991118 4272 ts_tablet_manager.cc:1428] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260812 16:34:54.995422 4402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.983309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:48925 (local address 127.19.238.126:46389)
0812 16:34:54.983542 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:54.983551 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:54.983593 (+ 42us) server_negotiation.cc:407] Connection header received
0812 16:34:54.984333 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:54.984338 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:54.984382 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:54.984542 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:54.985176 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.986130 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.987394 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.988092 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.988195 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.991567 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:54.991571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:54.991572 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:54.991584 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:54.993481 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:54.993944 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:54.993950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:54.993950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:54.994055 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:54.994367 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:54.994368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:54.994369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:54.994638 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:54.994741 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:54.995064 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 16:34:54.995210 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":101,"threads_started":1}
I20260812 16:34:54.995422 4401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.983204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:48925)
0812 16:34:54.983397 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:54.983441 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:54.984102 (+ 661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:54.984572 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:54.984576 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:54.984861 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:54.985051 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:54.985057 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.986261 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.986262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:54.987301 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:54.987305 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:54.988296 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:54.988297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:54.988453 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:54.989166 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:54.989175 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:54.991407 (+ 2232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:54.993582 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:54.993592 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:54.993594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:54.993846 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:54.994141 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:54.994142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:54.994142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:54.994269 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:54.994717 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:54.994726 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:54.994879 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 16:34:54.995006 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 16:34:54.995211 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":96,"threads_started":1}
I20260812 16:34:54.996673 4398 heartbeater.cc:347] Connected to a master server at 127.19.238.126:46389
I20260812 16:34:54.996810 4398 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:54.997015 4398 heartbeater.cc:511] Master 127.19.238.126:46389 requested a full tablet report, sending...
I20260812 16:34:54.997794 4179 ts_manager.cc:194] Registered new tserver with Master: 37ca7c03644b497d8cdc2fcaa310218b (127.19.238.65:45577)
I20260812 16:34:54.998996 4179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:48925
I20260812 16:34:54.999981 4398 heartbeater.cc:503] Master 127.19.238.126:46389 was elected leader, sending a full tablet report...
I20260812 16:34:55.004707 4405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.998447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:45577 (local address 127.0.0.1:33528)
0812 16:34:55.000834 (+ 2387us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:55.000896 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:55.000988 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:55.001415 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:55.001418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:55.001421 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:34:55.001489 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:55.001492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.002383 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.002384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:55.003502 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:55.003506 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.004326 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.004327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:55.004435 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.004450 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 16:34:55.004550 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 16:34:55.004623 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2350,"thread_start_us":107,"threads_started":1}
I20260812 16:34:55.004707 4406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:54.998725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33528 (local address 127.19.238.65:45577)
0812 16:34:55.001164 (+ 2439us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:55.001170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:55.001182 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:34:55.001284 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:55.001286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:55.001304 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:55.001398 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:34:55.001605 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.002288 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.003607 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.004229 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.004322 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.004391 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:55.004568 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 16:34:55.004618 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2396,"thread_start_us":74,"threads_started":1}
I20260812 16:34:55.005288 4322 tablet_service.cc:1514] Processing DeleteTablet for tablet f81d78092616454493598626f23dc480 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:34:54 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:33528
I20260812 16:34:55.005924 4407 tablet_replica.cc:330] stopping tablet replica
I20260812 16:34:55.006062 4407 raft_consensus.cc:2229] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:34:55.006138 4407 raft_consensus.cc:2258] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:34:55.006510 4407 ts_tablet_manager.cc:1905] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:34:55.009575 4407 ts_tablet_manager.cc:1918] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:34:55.009678 4407 log.cc:1196] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/wal/wals/f81d78092616454493598626f23dc480
I20260812 16:34:55.009939 4407 ts_tablet_manager.cc:1939] T f81d78092616454493598626f23dc480 P 37ca7c03644b497d8cdc2fcaa310218b: Deleting consensus metadata
I20260812 16:34:55.010715 4161 catalog_manager.cc:4841] TS 37ca7c03644b497d8cdc2fcaa310218b (127.19.238.65:45577): tablet f81d78092616454493598626f23dc480 (table test-table [id=8a9b97e61d854d49a665267861dc4626]) successfully deleted
W20260812 16:34:55.078701 4403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:55.078943 4403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:55.078969 4403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:55.080283 4403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:55.080336 4403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:55.080365 4403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:34:55.081996 4403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:43677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:41157
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=37955
--tserver_master_addrs=127.19.238.126:46389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:55.082302 4403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:55.082567 4403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:55.088121 4412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:55.088153 4413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:55.088224 4403 server_base.cc:1034] running on GCE node
W20260812 16:34:55.088128 4415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:55.088708 4403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:55.089031 4403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:55.090233 4403 hybrid_clock.cc:648] HybridClock initialized: now 1786552495090202 us; error 33 us; skew 500 ppm
Aug 12 16:34:55 dist-test-slave-mvvj krb5kdc[3529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552495, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:55.093134 4403 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:34:55.093604 4403 webserver.cc:460] Webserver started at http://127.19.238.66:37955/ using document root <none> and password file <none>
I20260812 16:34:55.093839 4403 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:55.093902 4403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:55.095420 4403 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:34:55.096210 4423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:55.096514 4403 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:34:55.096607 4403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "1466609a58564979844b45740b1b65ca"
format_stamp: "Formatted at 2026-08-12 16:34:54 on dist-test-slave-mvvj"
server_key: "9d5c66a6216a18dd8d5b9db094744ab7"
server_key_iv: "64e8343c7c1f9ca05ea6e82914dd5ea9"
server_key_version: "encryptionkey@0"
I20260812 16:34:55.096985 4403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:55.108794 4403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:55.111264 4403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:55.111394 4403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:55.111608 4403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:34:55.112000 4403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:34:55.112039 4403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:55.112083 4403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:34:55.112110 4403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:55.118384 4403 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:41157
I20260812 16:34:55.118451 4558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:41157 every 8 connection(s)
I20260812 16:34:55.118808 4403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
Aug 12 16:34:55 dist-test-slave-mvvj krb5kdc[3529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552495, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:34:55.125627 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 4403
I20260812 16:34:55.125890 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:37851
--local_ip_for_outbound_sockets=127.19.238.67
--tserver_master_addrs=127.19.238.126:46389
--webserver_port=40213
--webserver_interface=127.19.238.67
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:43677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:34:55.129500 4402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.119654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:32917 (local address 127.19.238.126:46389)
0812 16:34:55.119720 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:55.119722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:55.120026 (+ 304us) server_negotiation.cc:407] Connection header received
0812 16:34:55.120629 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:55.120631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:55.120649 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:55.120703 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:55.121735 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.122249 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.123325 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.123544 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.123624 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.125969 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:55.125971 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:55.125971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:55.125988 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:55.127745 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:55.128326 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:55.128327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:55.128328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:55.128384 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:55.128691 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:55.128691 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:55.128692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:55.128805 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:55.128854 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:55.129240 (+ 386us) server_negotiation.cc:299] Negotiation successful
0812 16:34:55.129348 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 16:34:55.129534 4561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.119630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:32917)
0812 16:34:55.119819 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:55.119911 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:55.120489 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:55.120777 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:55.120782 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:55.121391 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:55.121606 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:55.121611 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.122352 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.122353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:55.123220 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:55.123222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.123634 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.123635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:55.123778 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.124393 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:55.124402 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:55.125869 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:55.127872 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:55.127882 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:55.127884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:55.128233 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:55.128449 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:55.128450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:55.128450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:55.128612 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:55.128870 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:55.128879 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:55.129071 (+ 192us) client_negotiation.cc:769] Sending connection context
0812 16:34:55.129179 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 16:34:55.129356 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260812 16:34:55.130247 4559 heartbeater.cc:347] Connected to a master server at 127.19.238.126:46389
I20260812 16:34:55.130359 4559 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:55.130571 4559 heartbeater.cc:511] Master 127.19.238.126:46389 requested a full tablet report, sending...
I20260812 16:34:55.131011 4179 ts_manager.cc:194] Registered new tserver with Master: 1466609a58564979844b45740b1b65ca (127.19.238.66:41157)
I20260812 16:34:55.131433 4179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:32917
W20260812 16:34:55.211606 4562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:55.211876 4562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:55.211934 4562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:55.213112 4562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:55.213156 4562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:55.213183 4562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:34:55.214676 4562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:43677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:37851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=40213
--tserver_master_addrs=127.19.238.126:46389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:55.214952 4562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:55.215214 4562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:55.220348 4568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:55.220356 4567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:55.220356 4571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:55.220764 4562 server_base.cc:1034] running on GCE node
I20260812 16:34:55.220997 4562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:55.221246 4562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:55.222390 4562 hybrid_clock.cc:648] HybridClock initialized: now 1786552495222367 us; error 37 us; skew 500 ppm
Aug 12 16:34:55 dist-test-slave-mvvj krb5kdc[3529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552495, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:55.224932 4562 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:34:55.225425 4562 webserver.cc:460] Webserver started at http://127.19.238.67:40213/ using document root <none> and password file <none>
I20260812 16:34:55.225682 4562 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:55.225771 4562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:55.227176 4562 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:34:55.227759 4578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:55.227964 4562 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:34:55.228080 4562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7"
format_stamp: "Formatted at 2026-08-12 16:34:54 on dist-test-slave-mvvj"
server_key: "79fb17f6ba27d64d1793bf63467ee5dc"
server_key_iv: "146a360e71ce92f32dcefe5eac702919"
server_key_version: "encryptionkey@0"
I20260812 16:34:55.228444 4562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:55.249131 4562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:55.251498 4562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:55.251700 4562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:55.252048 4562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:34:55.252405 4562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:34:55.252466 4562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:55.252511 4562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:34:55.252537 4562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:55.259361 4562 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:37851
I20260812 16:34:55.259404 4710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:37851 every 8 connection(s)
I20260812 16:34:55.259887 4562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:34:55.263149 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 4562
Aug 12 16:34:55 dist-test-slave-mvvj krb5kdc[3529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552495, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:34:55.270499 4715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.263667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35776 (local address 127.19.238.126:46389)
0812 16:34:55.263915 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:55.263917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:55.263944 (+ 27us) server_negotiation.cc:407] Connection header received
0812 16:34:55.264051 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:55.264052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:55.264063 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:55.264155 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:55.264778 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.265261 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.266033 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.266321 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.266392 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.267798 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:55.267799 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:55.267800 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:55.267806 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:55.269260 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:55.269640 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:55.269641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:55.269641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:55.269699 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:55.269918 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:55.269919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:55.269919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:55.270036 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:55.270241 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:55.270278 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 16:34:55.270363 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":102,"threads_started":1}
I20260812 16:34:55.270603 4402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.260755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:53657 (local address 127.19.238.126:46389)
0812 16:34:55.260817 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:55.260819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:55.261072 (+ 253us) server_negotiation.cc:407] Connection header received
0812 16:34:55.261608 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:55.261609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:55.261621 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:55.261677 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:55.262693 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.263193 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.264565 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.264763 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.264841 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.267328 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:55.267330 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:55.267330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:55.267339 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:55.269126 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:55.269574 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:55.269575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:55.269576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:55.269636 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:55.269894 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:55.269895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:55.269896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:55.270018 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:55.270089 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:55.270382 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 16:34:55.270472 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 16:34:55.270828 4713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.260743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:53657)
0812 16:34:55.260915 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:55.260954 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:55.261496 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:55.261738 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:55.261741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:55.262376 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:55.262592 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:55.262597 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.263315 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.263318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:55.264432 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:55.264435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.264868 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.264870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:55.265007 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.265578 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:55.265587 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:55.267214 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:55.269228 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:55.269237 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:55.269239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:55.269484 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:55.269703 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:55.269703 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:55.269704 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:55.269821 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:55.270093 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:55.270101 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:55.270250 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 16:34:55.270363 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 16:34:55.270539 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":100,"threads_started":1}
I20260812 16:34:55.271411 4711 heartbeater.cc:347] Connected to a master server at 127.19.238.126:46389
I20260812 16:34:55.271520 4711 heartbeater.cc:464] Registering TS with master...
I20260812 16:34:55.271708 4711 heartbeater.cc:511] Master 127.19.238.126:46389 requested a full tablet report, sending...
I20260812 16:34:55.272135 4178 ts_manager.cc:194] Registered new tserver with Master: caa889dc6e294f9fa9b6df0fb0c1d1e7 (127.19.238.67:37851)
I20260812 16:34:55.272475 4178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:53657
I20260812 16:34:55.282670 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:34:55.285835 4715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.283124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35790 (local address 127.19.238.126:46389)
0812 16:34:55.283177 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:55.283180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:55.283227 (+ 47us) server_negotiation.cc:407] Connection header received
0812 16:34:55.283332 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:55.283333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:55.283374 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:55.283450 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:34:55.283610 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.284254 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.285103 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.285299 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.285349 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.285501 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:34:55.285605 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:34:55.285669 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:55.285704 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 16:34:55.285724 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 16:34:55.286191 4178 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:35790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:34:55.287811 4178 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:34:55.290692 4322 tablet_service.cc:1467] Processing CreateTablet for tablet dc0f1c03be6e4dcab7e6cb32b7724e5e (DEFAULT_TABLE table=test-table [id=1dc8aa4891214775b59959dda20712e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:34:55.290941 4322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc0f1c03be6e4dcab7e6cb32b7724e5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:34:55.292677 4272 tablet_bootstrap.cc:492] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b: Bootstrap starting.
I20260812 16:34:55.292981 4272 tablet_bootstrap.cc:654] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b: Neither blocks nor log segments found. Creating new log.
I20260812 16:34:55.293582 4272 tablet_bootstrap.cc:492] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b: No bootstrap required, opened a new log
I20260812 16:34:55.293654 4272 ts_tablet_manager.cc:1397] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 16:34:55.293774 4272 raft_consensus.cc:348] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:55.293893 4272 raft_consensus.cc:374] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:34:55.293951 4272 raft_consensus.cc:729] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37ca7c03644b497d8cdc2fcaa310218b, State: Initialized, Role: FOLLOWER
I20260812 16:34:55.294005 4272 consensus_queue.cc:260] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:55.294116 4272 ts_tablet_manager.cc:1428] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:55.294591 4405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.290056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:41157 (local address 127.0.0.1:55334)
0812 16:34:55.290108 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:55.290120 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:55.290160 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:55.290616 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:55.290617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:55.290619 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:34:55.290665 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:55.290667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.291985 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.291987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:55.293109 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:55.293112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.294346 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.294347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:55.294419 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.294423 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:34:55.294455 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 16:34:55.294509 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 16:34:55.295028 4719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.290217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55334 (local address 127.19.238.66:41157)
0812 16:34:55.290405 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:55.290411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:55.290424 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:34:55.290476 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:55.290486 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:55.290511 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:55.290612 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:34:55.290744 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.291792 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.293404 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.294241 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.294727 (+ 486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.294813 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:55.294859 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 16:34:55.294908 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":104,"threads_started":1}
I20260812 16:34:55.295157 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.290168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:37851 (local address 127.0.0.1:32870)
0812 16:34:55.290277 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:55.290286 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:55.290324 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:55.291060 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:55.291062 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:55.291064 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:34:55.291109 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:55.291111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.292474 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.292475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:55.293821 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:55.293824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.294876 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.294877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:55.294945 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.294949 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:34:55.295018 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 16:34:55.295066 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":77,"threads_started":1}
I20260812 16:34:55.295459 4721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.290550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32870 (local address 127.19.238.67:37851)
0812 16:34:55.290790 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:55.290794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:55.290807 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:34:55.290849 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:55.290851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:55.290868 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:55.290954 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:34:55.291186 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.292355 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.293931 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.294784 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.294865 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.294918 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:55.295051 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 16:34:55.295376 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":116,"threads_started":1}
I20260812 16:34:55.295874 4479 tablet_service.cc:1467] Processing CreateTablet for tablet dc0f1c03be6e4dcab7e6cb32b7724e5e (DEFAULT_TABLE table=test-table [id=1dc8aa4891214775b59959dda20712e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:34:55.296283 4479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc0f1c03be6e4dcab7e6cb32b7724e5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:34:55.296245 4633 tablet_service.cc:1467] Processing CreateTablet for tablet dc0f1c03be6e4dcab7e6cb32b7724e5e (DEFAULT_TABLE table=test-table [id=1dc8aa4891214775b59959dda20712e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:34:55.296607 4633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc0f1c03be6e4dcab7e6cb32b7724e5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:34:55.300410 4725 tablet_bootstrap.cc:492] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7: Bootstrap starting.
I20260812 16:34:55.300606 4724 tablet_bootstrap.cc:492] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca: Bootstrap starting.
I20260812 16:34:55.301249 4724 tablet_bootstrap.cc:654] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca: Neither blocks nor log segments found. Creating new log.
I20260812 16:34:55.301272 4725 tablet_bootstrap.cc:654] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7: Neither blocks nor log segments found. Creating new log.
I20260812 16:34:55.301899 4725 log.cc:824] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7: Log is configured to *not* fsync() on all Append() calls
I20260812 16:34:55.301899 4724 log.cc:824] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca: Log is configured to *not* fsync() on all Append() calls
I20260812 16:34:55.302858 4724 tablet_bootstrap.cc:492] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca: No bootstrap required, opened a new log
I20260812 16:34:55.302865 4725 tablet_bootstrap.cc:492] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7: No bootstrap required, opened a new log
I20260812 16:34:55.302939 4724 ts_tablet_manager.cc:1397] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:34:55.302939 4725 ts_tablet_manager.cc:1397] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:34:55.304978 4725 raft_consensus.cc:348] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:55.304968 4724 raft_consensus.cc:348] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:55.305157 4724 raft_consensus.cc:374] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:34:55.305157 4725 raft_consensus.cc:374] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:34:55.305205 4724 raft_consensus.cc:729] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1466609a58564979844b45740b1b65ca, State: Initialized, Role: FOLLOWER
I20260812 16:34:55.305205 4725 raft_consensus.cc:729] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caa889dc6e294f9fa9b6df0fb0c1d1e7, State: Initialized, Role: FOLLOWER
I20260812 16:34:55.305336 4724 consensus_queue.cc:260] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:55.305336 4725 consensus_queue.cc:260] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:55.305657 4725 ts_tablet_manager.cc:1428] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:34:55.305657 4724 ts_tablet_manager.cc:1428] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:34:55.305692 4711 heartbeater.cc:503] Master 127.19.238.126:46389 was elected leader, sending a full tablet report...
I20260812 16:34:55.305771 4559 heartbeater.cc:503] Master 127.19.238.126:46389 was elected leader, sending a full tablet report...
I20260812 16:34:55.309605 4728 raft_consensus.cc:482] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:34:55.309721 4728 raft_consensus.cc:504] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:55.310146 4728 leader_election.cc:288] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caa889dc6e294f9fa9b6df0fb0c1d1e7 (127.19.238.67:37851), 37ca7c03644b497d8cdc2fcaa310218b (127.19.238.65:45577)
I20260812 16:34:55.311563 4404 raft_consensus.cc:482] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:34:55.311672 4404 raft_consensus.cc:504] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:55.312021 4404 leader_election.cc:288] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1466609a58564979844b45740b1b65ca (127.19.238.66:41157), caa889dc6e294f9fa9b6df0fb0c1d1e7 (127.19.238.67:37851)
I20260812 16:34:55.314047 4561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.310283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:37851 (local address 127.19.238.66:33877)
0812 16:34:55.310350 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:55.310363 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:55.310437 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:55.310768 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:55.310769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:55.310771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:34:55.310842 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:55.310844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.311719 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.311720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:55.312979 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:55.312983 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.313757 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.313758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:55.313841 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.313845 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:34:55.313912 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 16:34:55.313957 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":37}
I20260812 16:34:55.314196 4406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.310573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:38801 (local address 127.19.238.65:45577)
0812 16:34:55.310640 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:55.310643 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:55.310791 (+ 148us) server_negotiation.cc:407] Connection header received
0812 16:34:55.310849 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:55.310850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:55.310859 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:55.310916 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:34:55.311079 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.311804 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.313254 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.313820 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.313866 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.313874 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:55.314087 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 16:34:55.314131 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 16:34:55.314353 4721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.310562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:33877 (local address 127.19.238.67:37851)
0812 16:34:55.310620 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:55.310622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:55.310630 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:34:55.310656 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:55.310657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:55.310666 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:55.310714 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:34:55.310933 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.311622 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.313086 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.313637 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.313711 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.313722 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:55.314225 (+ 503us) server_negotiation.cc:299] Negotiation successful
0812 16:34:55.314270 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 16:34:55.314704 4655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0f1c03be6e4dcab7e6cb32b7724e5e" candidate_uuid: "1466609a58564979844b45740b1b65ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" is_pre_election: true
I20260812 16:34:55.314869 4655 raft_consensus.cc:2454] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1466609a58564979844b45740b1b65ca in term 0.
I20260812 16:34:55.315142 4427 leader_election.cc:302] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1466609a58564979844b45740b1b65ca, caa889dc6e294f9fa9b6df0fb0c1d1e7; no voters:
I20260812 16:34:55.315263 4728 raft_consensus.cc:2781] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:34:55.315297 4728 raft_consensus.cc:482] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:34:55.315318 4728 raft_consensus.cc:3037] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:34:55.315737 4730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.310344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:45577 (local address 127.19.238.66:38801)
0812 16:34:55.310715 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:55.310725 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:55.310791 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:55.310919 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:55.310920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:55.310921 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:34:55.311001 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:55.311003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.311934 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.311935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:55.313160 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:55.313162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.313932 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.313933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:55.314004 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.314009 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:34:55.315602 (+ 1593us) client_negotiation.cc:240] Negotiation successful
0812 16:34:55.315652 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":326,"threads_started":1}
I20260812 16:34:55.316037 4343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0f1c03be6e4dcab7e6cb32b7724e5e" candidate_uuid: "1466609a58564979844b45740b1b65ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ca7c03644b497d8cdc2fcaa310218b" is_pre_election: true
I20260812 16:34:55.316146 4343 raft_consensus.cc:2454] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1466609a58564979844b45740b1b65ca in term 0.
I20260812 16:34:55.316351 4728 raft_consensus.cc:504] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:55.316635 4655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0f1c03be6e4dcab7e6cb32b7724e5e" candidate_uuid: "1466609a58564979844b45740b1b65ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7"
I20260812 16:34:55.316635 4343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0f1c03be6e4dcab7e6cb32b7724e5e" candidate_uuid: "1466609a58564979844b45740b1b65ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ca7c03644b497d8cdc2fcaa310218b"
I20260812 16:34:55.316710 4719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.312787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:42467 (local address 127.19.238.66:41157)
0812 16:34:55.312824 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:55.312826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:55.312921 (+ 95us) server_negotiation.cc:407] Connection header received
0812 16:34:55.313021 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:55.313023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:55.313032 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:55.313059 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:34:55.313310 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.313958 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.315591 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.316118 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.316195 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.316204 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:55.316608 (+ 404us) server_negotiation.cc:299] Negotiation successful
0812 16:34:55.316645 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:34:55.316711 4343 raft_consensus.cc:3037] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:34:55.316711 4655 raft_consensus.cc:3037] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:34:55.316958 4732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.312619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:41157 (local address 127.19.238.65:42467)
0812 16:34:55.312871 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:55.312880 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:55.312942 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:55.313148 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:55.313148 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:55.313150 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:34:55.313221 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:55.313223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.314059 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.314060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:55.315522 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:55.315524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.316401 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.316403 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:55.316488 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.316492 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:34:55.316817 (+ 325us) client_negotiation.cc:240] Negotiation successful
0812 16:34:55.316863 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":79,"threads_started":1}
I20260812 16:34:55.317217 4505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0f1c03be6e4dcab7e6cb32b7724e5e" candidate_uuid: "37ca7c03644b497d8cdc2fcaa310218b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1466609a58564979844b45740b1b65ca" is_pre_election: true
I20260812 16:34:55.317327 4505 raft_consensus.cc:2379] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 37ca7c03644b497d8cdc2fcaa310218b in current term 1: Already voted for candidate 1466609a58564979844b45740b1b65ca in this term.
I20260812 16:34:55.317610 4655 raft_consensus.cc:2454] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1466609a58564979844b45740b1b65ca in term 1.
I20260812 16:34:55.317606 4401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.312207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:37851 (local address 127.19.238.65:42081)
0812 16:34:55.312295 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:55.312304 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:55.312342 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:55.314417 (+ 2075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:55.314417 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:55.314419 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:34:55.314468 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:55.314469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.315450 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.315451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:55.316519 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:55.316521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.317387 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.317388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:55.317467 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.317471 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:34:55.317505 (+ 34us) client_negotiation.cc:240] Negotiation successful
0812 16:34:55.317546 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 16:34:55.317631 4728 leader_election.cc:288] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [CANDIDATE]: Term 1 election: Requested vote from peers caa889dc6e294f9fa9b6df0fb0c1d1e7 (127.19.238.67:37851), 37ca7c03644b497d8cdc2fcaa310218b (127.19.238.65:45577)
I20260812 16:34:55.317610 4343 raft_consensus.cc:2454] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1466609a58564979844b45740b1b65ca in term 1.
I20260812 16:34:55.317782 4427 leader_election.cc:302] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1466609a58564979844b45740b1b65ca, caa889dc6e294f9fa9b6df0fb0c1d1e7; no voters:
I20260812 16:34:55.317857 4728 raft_consensus.cc:2781] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:34:55.317934 4731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.312411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:42081 (local address 127.19.238.67:37851)
0812 16:34:55.314274 (+ 1863us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:55.314276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:55.314284 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:34:55.314309 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:55.314309 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:55.314319 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:55.314528 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:34:55.314562 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.315360 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.316612 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.317286 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.317798 (+ 512us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.317808 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:55.317824 (+ 16us) server_negotiation.cc:299] Negotiation successful
0812 16:34:55.317861 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1836,"thread_start_us":96,"threads_started":1}
I20260812 16:34:55.318043 4728 raft_consensus.cc:686] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [term 1 LEADER]: Becoming Leader. State: Replica: 1466609a58564979844b45740b1b65ca, State: Running, Role: LEADER
I20260812 16:34:55.318112 4655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0f1c03be6e4dcab7e6cb32b7724e5e" candidate_uuid: "37ca7c03644b497d8cdc2fcaa310218b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" is_pre_election: true
I20260812 16:34:55.318135 4728 consensus_queue.cc:237] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:55.318222 4655 raft_consensus.cc:2379] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 37ca7c03644b497d8cdc2fcaa310218b in current term 1: Already voted for candidate 1466609a58564979844b45740b1b65ca in this term.
I20260812 16:34:55.318477 4269 leader_election.cc:302] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 37ca7c03644b497d8cdc2fcaa310218b; no voters: 1466609a58564979844b45740b1b65ca, caa889dc6e294f9fa9b6df0fb0c1d1e7
I20260812 16:34:55.318620 4404 raft_consensus.cc:2726] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 16:34:55.319092 4179 catalog_manager.cc:5495] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 1466609a58564979844b45740b1b65ca (127.19.238.66). New cstate: current_term: 1 leader_uuid: "1466609a58564979844b45740b1b65ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } health_report { overall_health: UNKNOWN } } }
I20260812 16:34:55.326133 4719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.323761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55346 (local address 127.19.238.66:41157)
0812 16:34:55.323811 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:55.323813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:55.323822 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:34:55.323888 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:55.323889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:55.323909 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:55.323980 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:34:55.324080 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.324592 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.325328 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:55.325550 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:55.325627 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:55.325769 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:34:55.325877 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:34:55.325969 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:55.326008 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 16:34:55.326029 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 16:34:55.328166 4343 raft_consensus.cc:1261] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Refusing update from remote peer 1466609a58564979844b45740b1b65ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:34:55.328187 4655 raft_consensus.cc:1261] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 1 FOLLOWER]: Refusing update from remote peer 1466609a58564979844b45740b1b65ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:34:55.328547 4728 consensus_queue.cc:1035] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:34:55.328688 4733 consensus_queue.cc:1035] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:34:55.331380 4737 mvcc.cc:205] Tried to move back new op lower bound from 7317719020860862464 to 7317719020823867392. Current Snapshot: MvccSnapshot[applied={T|T < 7317719020860862464}]
I20260812 16:34:55.333335 4179 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:35790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:34:55.333417 4179 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:35790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:34:55.334154 4179 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d: Sending DeleteTablet for 3 replicas of tablet dc0f1c03be6e4dcab7e6cb32b7724e5e
I20260812 16:34:55.334470 4322 tablet_service.cc:1514] Processing DeleteTablet for tablet dc0f1c03be6e4dcab7e6cb32b7724e5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:34:55 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:33528
I20260812 16:34:55.334502 4479 tablet_service.cc:1514] Processing DeleteTablet for tablet dc0f1c03be6e4dcab7e6cb32b7724e5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:34:55 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:55334
I20260812 16:34:55.334579 4407 tablet_replica.cc:330] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b: stopping tablet replica
I20260812 16:34:55.334640 4407 raft_consensus.cc:2229] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:34:55.334625 4633 tablet_service.cc:1514] Processing DeleteTablet for tablet dc0f1c03be6e4dcab7e6cb32b7724e5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:34:55 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:32870
I20260812 16:34:55.334686 4407 raft_consensus.cc:2258] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:34:55.334926 4750 tablet_replica.cc:330] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7: stopping tablet replica
I20260812 16:34:55.334926 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 4223
I20260812 16:34:55.335027 4750 raft_consensus.cc:2229] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:34:55.335126 4750 raft_consensus.cc:2258] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:34:55.335342 4748 tablet_replica.cc:330] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca: stopping tablet replica
I20260812 16:34:55.335430 4748 raft_consensus.cc:2229] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:34:55.335598 4748 raft_consensus.cc:2258] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:34:55.341224 4748 ts_tablet_manager.cc:1905] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:34:55.343310 4748 ts_tablet_manager.cc:1918] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:34:55.343451 4748 log.cc:1196] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/dc0f1c03be6e4dcab7e6cb32b7724e5e
I20260812 16:34:55.343775 4748 ts_tablet_manager.cc:1939] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 1466609a58564979844b45740b1b65ca: Deleting consensus metadata
I20260812 16:34:55.344340 4163 catalog_manager.cc:4841] TS 1466609a58564979844b45740b1b65ca (127.19.238.66:41157): tablet dc0f1c03be6e4dcab7e6cb32b7724e5e (table test-table [id=1dc8aa4891214775b59959dda20712e7]) successfully deleted
W20260812 16:34:55.346391 4161 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.238.65:45577 (error 108)
I20260812 16:34:55.346998 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.346846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:45577 (local address 127.0.0.1:33530)
0812 16:34:55.346900 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:55.346944 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:45577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 16:34:55.347219 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 4403
I20260812 16:34:55.347362 4750 ts_tablet_manager.cc:1905] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 16:34:55.347136 4161 catalog_manager.cc:4584] TS 37ca7c03644b497d8cdc2fcaa310218b (127.19.238.65:45577): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dc0f1c03be6e4dcab7e6cb32b7724e5e: Network error: Client connection negotiation failed: client connection to 127.19.238.65:45577: connect: Connection refused (error 111)
I20260812 16:34:55.349143 4750 ts_tablet_manager.cc:1918] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:34:55.349226 4750 log.cc:1196] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/dc0f1c03be6e4dcab7e6cb32b7724e5e
I20260812 16:34:55.349524 4750 ts_tablet_manager.cc:1939] T dc0f1c03be6e4dcab7e6cb32b7724e5e P caa889dc6e294f9fa9b6df0fb0c1d1e7: Deleting consensus metadata
I20260812 16:34:55.350033 4164 catalog_manager.cc:4841] TS caa889dc6e294f9fa9b6df0fb0c1d1e7 (127.19.238.67:37851): tablet dc0f1c03be6e4dcab7e6cb32b7724e5e (table test-table [id=1dc8aa4891214775b59959dda20712e7]) successfully deleted
I20260812 16:34:55.355582 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 4562
I20260812 16:34:55.365243 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 4141
I20260812 16:34:55.373200 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:46389
--webserver_interface=127.19.238.126
--webserver_port=40093
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:43677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:46389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:34:55.460443 4752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:55.460769 4752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:55.460809 4752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:55.462610 4752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:34:55.462664 4752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:55.462679 4752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:55.462692 4752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:34:55.462705 4752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:34:55.464781 4752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:43677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:46389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:46389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=40093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:55.465090 4752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:55.465368 4752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:55.470476 4759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:55.470479 4760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:55.470479 4763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:55.470731 4752 server_base.cc:1034] running on GCE node
I20260812 16:34:55.471058 4752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:55.471336 4752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:55.472486 4752 hybrid_clock.cc:648] HybridClock initialized: now 1786552495472469 us; error 29 us; skew 500 ppm
Aug 12 16:34:55 dist-test-slave-mvvj krb5kdc[3529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552495, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:55.475203 4752 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:34:55.475637 4752 webserver.cc:460] Webserver started at http://127.19.238.126:40093/ using document root <none> and password file <none>
I20260812 16:34:55.475898 4752 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:55.475957 4752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:55.477349 4752 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:34:55.478050 4772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:55.478216 4752 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:34:55.478314 4752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "3ed0d01bd8c145b199d22029262a0a9d"
format_stamp: "Formatted at 2026-08-12 16:34:53 on dist-test-slave-mvvj"
server_key: "11596787d5c9a28766ca2c0a0d761e90"
server_key_iv: "c89cadbb89e66b4389516812d884a817"
server_key_version: "encryptionkey@0"
I20260812 16:34:55.478668 4752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:55.495762 4752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:55.498113 4752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:55.498270 4752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:55.502604 4752 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:46389
I20260812 16:34:55.502643 4834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:46389 every 8 connection(s)
I20260812 16:34:55.503051 4752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:34:55.504818 4835 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:34:55.505513 4835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d: Bootstrap starting.
I20260812 16:34:55.509148 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 4752
I20260812 16:34:55.509466 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:45577
--local_ip_for_outbound_sockets=127.19.238.65
--tserver_master_addrs=127.19.238.126:46389
--webserver_port=43867
--webserver_interface=127.19.238.65
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:43677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:34:55.514678 4835 log.cc:824] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:34:55.517113 4835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:34:55.517463 4835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d: Bootstrap complete.
I20260812 16:34:55.519922 4835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed0d01bd8c145b199d22029262a0a9d" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46389 } }
I20260812 16:34:55.520213 4835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed0d01bd8c145b199d22029262a0a9d, State: Initialized, Role: FOLLOWER
I20260812 16:34:55.520404 4835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed0d01bd8c145b199d22029262a0a9d" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46389 } }
I20260812 16:34:55.520493 4835 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:34:55.520553 4835 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:34:55.520609 4835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 2 FOLLOWER]: Advancing to term 3
I20260812 16:34:55.521668 4835 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed0d01bd8c145b199d22029262a0a9d" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46389 } }
I20260812 16:34:55.521798 4835 leader_election.cc:302] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ed0d01bd8c145b199d22029262a0a9d; no voters:
I20260812 16:34:55.522018 4835 leader_election.cc:288] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 16:34:55.522058 4839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 3 FOLLOWER]: Leader election won for term 3
I20260812 16:34:55.522177 4839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [term 3 LEADER]: Becoming Leader. State: Replica: 3ed0d01bd8c145b199d22029262a0a9d, State: Running, Role: LEADER
I20260812 16:34:55.522269 4839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed0d01bd8c145b199d22029262a0a9d" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46389 } }
I20260812 16:34:55.522313 4835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:34:55.522536 4839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ed0d01bd8c145b199d22029262a0a9d. Latest consensus state: current_term: 3 leader_uuid: "3ed0d01bd8c145b199d22029262a0a9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed0d01bd8c145b199d22029262a0a9d" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46389 } } }
I20260812 16:34:55.522619 4839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [sys.catalog]: This master's current role is: LEADER
I20260812 16:34:55.522541 4840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3ed0d01bd8c145b199d22029262a0a9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed0d01bd8c145b199d22029262a0a9d" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46389 } } }
I20260812 16:34:55.522791 4840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed0d01bd8c145b199d22029262a0a9d [sys.catalog]: This master's current role is: LEADER
I20260812 16:34:55.522822 4843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:34:55.523810 4843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:34:55.523989 4843 catalog_manager.cc:1259] Loaded cluster ID: d8f016f9ce7040478b698fb59f67b4b3
I20260812 16:34:55.524356 4843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 16:34:55.524709 4843 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f901a12d9ad google::LogMessage::Fail() at ??:0
@ 0x7f901a13088a google::LogMessage::SendToLog() at ??:0
@ 0x7f901a12d4de google::LogMessage::Flush() at ??:0
@ 0x7f901a12ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f902204e0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f90220578b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f901bffb0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f901bff0954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f901ce546db start_thread at ??:0
@ 0x7f9018c2b61f clone at ??:0
W20260812 16:34:55.611941 4837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:55.612162 4837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:55.612195 4837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:55.613425 4837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:55.613476 4837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:55.613505 4837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:34:55.615034 4837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:43677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:45577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=43867
--tserver_master_addrs=127.19.238.126:46389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:55.615309 4837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:55.615595 4837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:55.622082 4877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:55.622336 4837 server_base.cc:1034] running on GCE node
W20260812 16:34:55.622219 4880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:55.622130 4878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:55.622704 4837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:55.623004 4837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:55.624197 4837 hybrid_clock.cc:648] HybridClock initialized: now 1786552495624173 us; error 39 us; skew 500 ppm
Aug 12 16:34:55 dist-test-slave-mvvj krb5kdc[3529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552495, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:55.626973 4837 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:34:55.627516 4837 webserver.cc:460] Webserver started at http://127.19.238.65:43867/ using document root <none> and password file <none>
I20260812 16:34:55.627776 4837 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:55.627866 4837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:55.629359 4837 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:34:55.630155 4888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:55.630342 4837 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:34:55.630451 4837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "37ca7c03644b497d8cdc2fcaa310218b"
format_stamp: "Formatted at 2026-08-12 16:34:54 on dist-test-slave-mvvj"
server_key: "b596875b76f09b0e856d64c56f30755b"
server_key_iv: "9249a202c14803f39814718d2fde2569"
server_key_version: "encryptionkey@0"
I20260812 16:34:55.630822 4837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:55.645145 4837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:55.647488 4837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:55.647676 4837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:55.647989 4837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:34:55.648489 4897 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 16:34:55.650568 4837 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 16:34:55.650650 4837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 16:34:55.650698 4837 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 16:34:55.651427 4837 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 16:34:55.651504 4837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 16:34:55.651571 4897 tablet_bootstrap.cc:492] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b: Bootstrap starting.
I20260812 16:34:55.659462 4837 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:45577
I20260812 16:34:55.659548 5021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:45577 every 8 connection(s)
I20260812 16:34:55.659991 4837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:34:55.663868 4897 log.cc:824] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b: Log is configured to *not* fsync() on all Append() calls
I20260812 16:34:55.666328 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 4837
I20260812 16:34:55.666674 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:41157
--local_ip_for_outbound_sockets=127.19.238.66
--tserver_master_addrs=127.19.238.126:46389
--webserver_port=37955
--webserver_interface=127.19.238.66
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:43677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:34:55.667240 4897 tablet_bootstrap.cc:492] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:34:55.667707 4897 tablet_bootstrap.cc:492] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b: Bootstrap complete.
I20260812 16:34:55.669409 4897 ts_tablet_manager.cc:1397] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.008s
I20260812 16:34:55.670976 4897 raft_consensus.cc:348] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:55.671366 4897 raft_consensus.cc:729] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37ca7c03644b497d8cdc2fcaa310218b, State: Initialized, Role: FOLLOWER
I20260812 16:34:55.671577 4897 consensus_queue.cc:260] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:55.671948 4897 ts_tablet_manager.cc:1428] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260812 16:34:55.727634 5024 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:34:55.660754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:42569)
0812 16:34:55.660952 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:55.660994 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:55.661494 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:55.727400 (+ 65906us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":108,"threads_started":1}
W20260812 16:34:55.727917 5022 heartbeater.cc:644] Failed to heartbeat to 127.19.238.126:46389 (0 consecutive failures): Network error: Failed to ping master at 127.19.238.126:46389: Client connection negotiation failed: client connection to 127.19.238.126:46389: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 16:34:55.728288 5024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.728158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:36945)
0812 16:34:55.728218 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:55.728269 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 16:34:55.728726 5024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.728625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:55421)
0812 16:34:55.728680 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:55.728707 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 16:34:55.728875 5022 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 16:34:55.759565 5027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:55.759809 5027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:55.759892 5027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:55.761155 5027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:55.761207 5027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:55.761237 5027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:34:55.762885 5027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:43677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:41157
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=37955
--tserver_master_addrs=127.19.238.126:46389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:55.763182 5027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:55.763434 5027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:55.768870 5036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:55.768882 5034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:55.768877 5033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:55.769100 5027 server_base.cc:1034] running on GCE node
I20260812 16:34:55.769399 5027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:55.769667 5027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:55.770823 5027 hybrid_clock.cc:648] HybridClock initialized: now 1786552495770802 us; error 43 us; skew 500 ppm
Aug 12 16:34:55 dist-test-slave-mvvj krb5kdc[3529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552495, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:55.773573 5027 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:34:55.774096 5027 webserver.cc:460] Webserver started at http://127.19.238.66:37955/ using document root <none> and password file <none>
I20260812 16:34:55.774334 5027 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:55.774382 5027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:55.775897 5027 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:34:55.776574 5045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:55.776768 5027 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:34:55.776865 5027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "1466609a58564979844b45740b1b65ca"
format_stamp: "Formatted at 2026-08-12 16:34:54 on dist-test-slave-mvvj"
server_key: "9d5c66a6216a18dd8d5b9db094744ab7"
server_key_iv: "64e8343c7c1f9ca05ea6e82914dd5ea9"
server_key_version: "encryptionkey@0"
I20260812 16:34:55.777268 5027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:55.794430 5027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:55.796993 5027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:55.797164 5027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:55.797458 5027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:34:55.797798 5027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:34:55.797833 5027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:55.797879 5027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:34:55.797907 5027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:55.804620 5027 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:41157
I20260812 16:34:55.804698 5171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:41157 every 8 connection(s)
I20260812 16:34:55.805058 5027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:34:55.806188 5174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.805772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:47789)
0812 16:34:55.805947 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:55.806039 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
W20260812 16:34:55.806766 5172 heartbeater.cc:644] Failed to heartbeat to 127.19.238.126:46389 (0 consecutive failures): Network error: Failed to ping master at 127.19.238.126:46389: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
I20260812 16:34:55.807184 5174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.807070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:44007)
0812 16:34:55.807128 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:55.807163 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 16:34:55.807538 5174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.807444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:44161)
0812 16:34:55.807501 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:55.807524 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 16:34:55.807701 5172 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 16:34:55.814601 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 5027
I20260812 16:34:55.814839 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:37851
--local_ip_for_outbound_sockets=127.19.238.67
--tserver_master_addrs=127.19.238.126:46389
--webserver_port=40213
--webserver_interface=127.19.238.67
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:43677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:34:55.901117 5175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:34:55.901427 5175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:34:55.901471 5175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:34:55.902736 5175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:34:55.902808 5175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:34:55.902853 5175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:34:55.904480 5175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:43677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:37851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=40213
--tserver_master_addrs=127.19.238.126:46389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:34:55.904768 5175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:34:55.905030 5175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:34:55.910571 5182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:55.910620 5175 server_base.cc:1034] running on GCE node
W20260812 16:34:55.910581 5184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:34:55.910669 5181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:34:55.911058 5175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:34:55.911290 5175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:34:55.912431 5175 hybrid_clock.cc:648] HybridClock initialized: now 1786552495912419 us; error 61 us; skew 500 ppm
Aug 12 16:34:55 dist-test-slave-mvvj krb5kdc[3529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552495, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:34:55.915112 5175 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:34:55.915597 5175 webserver.cc:460] Webserver started at http://127.19.238.67:40213/ using document root <none> and password file <none>
I20260812 16:34:55.915879 5175 fs_manager.cc:359] Metadata directory not provided
I20260812 16:34:55.915941 5175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:34:55.917305 5175 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:34:55.917974 5195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:55.918179 5175 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:34:55.918285 5175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7"
format_stamp: "Formatted at 2026-08-12 16:34:54 on dist-test-slave-mvvj"
server_key: "79fb17f6ba27d64d1793bf63467ee5dc"
server_key_iv: "146a360e71ce92f32dcefe5eac702919"
server_key_version: "encryptionkey@0"
I20260812 16:34:55.918707 5175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:34:55.930601 5175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:34:55.932816 5175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:34:55.933004 5175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:34:55.933264 5175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:34:55.933602 5175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:34:55.933660 5175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:55.933706 5175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:34:55.933730 5175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:34:55.940095 5175 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:37851
I20260812 16:34:55.940145 5326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:37851 every 8 connection(s)
I20260812 16:34:55.940605 5175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:34:55.941187 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 5175
I20260812 16:34:55.941908 5329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.941435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:47791)
0812 16:34:55.941609 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:55.941709 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
W20260812 16:34:55.942298 5327 heartbeater.cc:644] Failed to heartbeat to 127.19.238.126:46389 (0 consecutive failures): Network error: Failed to ping master at 127.19.238.126:46389: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
I20260812 16:34:55.942612 5329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.942507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:38563)
0812 16:34:55.942558 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:55.942587 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 16:34:55.942896 5329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:55.942800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:43913)
0812 16:34:55.942849 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:55.942871 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 16:34:55.942999 5327 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 16:34:56.729739 5331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:56.729397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:53621)
0812 16:34:56.729636 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:56.729685 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":78,"threads_started":1}
I20260812 16:34:56.808357 5332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:56.808099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:60047)
0812 16:34:56.808246 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:56.808288 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":58,"threads_started":1}
I20260812 16:34:56.943737 5334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:56.943468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:49299)
0812 16:34:56.943634 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:56.943679 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
I20260812 16:34:57.363266 5335 raft_consensus.cc:482] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:34:57.363447 5335 raft_consensus.cc:504] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:57.363941 5335 leader_election.cc:288] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1466609a58564979844b45740b1b65ca (127.19.238.66:41157), caa889dc6e294f9fa9b6df0fb0c1d1e7 (127.19.238.67:37851)
Aug 12 16:34:57 dist-test-slave-mvvj krb5kdc[3529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552495, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.66@KRBTEST.COM
Aug 12 16:34:57 dist-test-slave-mvvj krb5kdc[3529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552495, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.67@KRBTEST.COM
I20260812 16:34:57.375991 5339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:57.364315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:60677 (local address 127.19.238.66:41157)
0812 16:34:57.364470 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:57.364481 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:57.364503 (+ 22us) server_negotiation.cc:407] Connection header received
0812 16:34:57.364973 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:57.364975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:57.365032 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:57.365229 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:57.365976 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:57.366945 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:57.368101 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:57.368541 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:57.368676 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:57.371690 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:57.371696 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:57.371696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:57.371711 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:57.373852 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:57.374320 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:57.374324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:57.374324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:57.374454 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:57.374799 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:57.374801 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:57.374801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:57.375197 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:57.375296 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:57.375669 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 16:34:57.375819 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260812 16:34:57.375996 5338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:57.364264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:54861 (local address 127.19.238.67:37851)
0812 16:34:57.364458 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 16:34:57.364472 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 16:34:57.364519 (+ 47us) server_negotiation.cc:407] Connection header received
0812 16:34:57.365021 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:34:57.365023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:34:57.365065 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:34:57.365234 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:34:57.365986 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:57.366932 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:57.368095 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:34:57.368545 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:34:57.368678 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:57.372197 (+ 3519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:34:57.372201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:34:57.372202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:34:57.372215 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:34:57.374065 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:57.374584 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:57.374587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:57.374587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:57.374742 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:34:57.374992 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:34:57.374993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:34:57.374994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:34:57.375324 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:34:57.375424 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 16:34:57.375687 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 16:34:57.375853 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":64,"threads_started":1}
I20260812 16:34:57.375981 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:57.364148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:37851 (local address 127.19.238.65:54861)
0812 16:34:57.364311 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:57.364324 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:57.364402 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:57.365262 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:57.365265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:57.365640 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:57.365840 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:57.365847 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:57.367061 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:57.367063 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:57.367991 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:57.367994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:57.368650 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:57.368652 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:57.368783 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:57.369474 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:57.369484 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:57.372063 (+ 2579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:57.374189 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:57.374191 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:57.374191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:57.374493 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:57.374835 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:57.374837 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:57.374837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:57.374917 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:57.375411 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:57.375412 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:57.375465 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 16:34:57.375605 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 16:34:57.375873 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":59,"thread_start_us":78,"threads_started":1}
I20260812 16:34:57.376120 5336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:57.364105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:41157 (local address 127.19.238.65:60677)
0812 16:34:57.364263 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:57.364277 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:34:57.364349 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:34:57.365262 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:34:57.365265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:34:57.365638 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:34:57.365840 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:57.365846 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:57.367103 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:57.367104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:57.367988 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:34:57.367991 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:34:57.368649 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:34:57.368650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:34:57.368789 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:34:57.369478 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:34:57.369489 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:34:57.371511 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:34:57.373970 (+ 2459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:57.373978 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:57.373980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:57.374222 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:57.374541 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:34:57.374542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:34:57.374542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:34:57.374721 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:34:57.375290 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:34:57.375299 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:34:57.375464 (+ 165us) client_negotiation.cc:769] Sending connection context
0812 16:34:57.375605 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 16:34:57.375957 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":186,"thread_start_us":60,"threads_started":1}
W20260812 16:34:57.376979 4894 leader_election.cc:341] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer caa889dc6e294f9fa9b6df0fb0c1d1e7 (127.19.238.67:37851): Not found: Tablet not found: dc0f1c03be6e4dcab7e6cb32b7724e5e
W20260812 16:34:57.377646 4893 leader_election.cc:341] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1466609a58564979844b45740b1b65ca (127.19.238.66:41157): Not found: Tablet not found: dc0f1c03be6e4dcab7e6cb32b7724e5e
I20260812 16:34:57.377717 4893 leader_election.cc:302] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 37ca7c03644b497d8cdc2fcaa310218b; no voters: 1466609a58564979844b45740b1b65ca, caa889dc6e294f9fa9b6df0fb0c1d1e7
I20260812 16:34:57.377844 5335 raft_consensus.cc:2726] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:34:57.730630 5336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:57.730419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:42493)
0812 16:34:57.730527 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:57.730582 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260812 16:34:57.809134 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:57.808878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:48855)
0812 16:34:57.809023 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:57.809080 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260812 16:34:57.944648 5342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:57.944403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:57543)
0812 16:34:57.944552 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:57.944600 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260812 16:34:58.731542 5343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:58.731262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:60069)
0812 16:34:58.731434 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:58.731484 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":79,"threads_started":1}
I20260812 16:34:58.810055 5344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:58.809750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:51475)
0812 16:34:58.809928 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:58.809997 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":76,"threads_started":1}
I20260812 16:34:58.945477 5345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:58.945181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:41747)
0812 16:34:58.945370 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:58.945422 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":67,"threads_started":1}
I20260812 16:34:59.200196 5346 raft_consensus.cc:482] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:34:59.200299 5346 raft_consensus.cc:504] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:34:59.200522 5346 leader_election.cc:288] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1466609a58564979844b45740b1b65ca (127.19.238.66:41157), caa889dc6e294f9fa9b6df0fb0c1d1e7 (127.19.238.67:37851)
W20260812 16:34:59.201013 4894 leader_election.cc:341] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer caa889dc6e294f9fa9b6df0fb0c1d1e7 (127.19.238.67:37851): Not found: Tablet not found: dc0f1c03be6e4dcab7e6cb32b7724e5e
W20260812 16:34:59.201090 4893 leader_election.cc:341] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1466609a58564979844b45740b1b65ca (127.19.238.66:41157): Not found: Tablet not found: dc0f1c03be6e4dcab7e6cb32b7724e5e
I20260812 16:34:59.201171 4893 leader_election.cc:302] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 37ca7c03644b497d8cdc2fcaa310218b; no voters: 1466609a58564979844b45740b1b65ca, caa889dc6e294f9fa9b6df0fb0c1d1e7
I20260812 16:34:59.201275 5346 raft_consensus.cc:2726] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:34:59.732452 5347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:59.732213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:46835)
0812 16:34:59.732357 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:59.732402 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
I20260812 16:34:59.810945 5348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:59.810676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:50541)
0812 16:34:59.810846 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:59.810896 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":74,"threads_started":1}
I20260812 16:34:59.946307 5351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:34:59.946062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:48619)
0812 16:34:59.946214 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 16:34:59.946259 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":76,"threads_started":1}
I20260812 16:35:00.733286 5352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:00.733060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:39105)
0812 16:35:00.733198 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:00.733238 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
I20260812 16:35:00.811789 5353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:00.811520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:52145)
0812 16:35:00.811688 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:00.811743 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":84,"threads_started":1}
I20260812 16:35:00.947146 5354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:00.946895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:47339)
0812 16:35:00.947052 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:00.947099 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":68,"threads_started":1}
I20260812 16:35:01.734241 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:01.733887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:60249)
0812 16:35:01.734136 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:01.734191 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":98,"threads_started":1}
I20260812 16:35:01.812877 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:01.812506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:42589)
0812 16:35:01.812764 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:01.812815 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":97,"threads_started":1}
I20260812 16:35:01.864797 5357 raft_consensus.cc:482] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:35:01.864904 5357 raft_consensus.cc:504] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:35:01.865106 5357 leader_election.cc:288] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1466609a58564979844b45740b1b65ca (127.19.238.66:41157), caa889dc6e294f9fa9b6df0fb0c1d1e7 (127.19.238.67:37851)
W20260812 16:35:01.865577 4893 leader_election.cc:341] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1466609a58564979844b45740b1b65ca (127.19.238.66:41157): Not found: Tablet not found: dc0f1c03be6e4dcab7e6cb32b7724e5e
W20260812 16:35:01.865698 4894 leader_election.cc:341] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer caa889dc6e294f9fa9b6df0fb0c1d1e7 (127.19.238.67:37851): Not found: Tablet not found: dc0f1c03be6e4dcab7e6cb32b7724e5e
I20260812 16:35:01.865762 4894 leader_election.cc:302] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 37ca7c03644b497d8cdc2fcaa310218b; no voters: 1466609a58564979844b45740b1b65ca, caa889dc6e294f9fa9b6df0fb0c1d1e7
I20260812 16:35:01.865857 5357 raft_consensus.cc:2726] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:35:01.948206 5358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:01.947958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:55807)
0812 16:35:01.948110 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:01.948157 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":68,"threads_started":1}
I20260812 16:35:02.735126 5359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:02.734894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:40029)
0812 16:35:02.735028 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:02.735073 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260812 16:35:02.813659 5360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:02.813396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:41679)
0812 16:35:02.813560 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:02.813607 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":73,"threads_started":1}
I20260812 16:35:02.949023 5361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:02.948765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:58685)
0812 16:35:02.948912 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:02.948975 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":74,"threads_started":1}
I20260812 16:35:03.736085 5362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:03.735711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:36281)
0812 16:35:03.735987 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:03.736035 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":78,"threads_started":1}
I20260812 16:35:03.814531 5363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:03.814264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:41045)
0812 16:35:03.814430 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:03.814475 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":68,"threads_started":1}
I20260812 16:35:03.949896 5364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:03.949585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:57001)
0812 16:35:03.949799 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:03.949849 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":67,"threads_started":1}
I20260812 16:35:04.737044 5365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:04.736699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:55311)
0812 16:35:04.736932 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:04.736985 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":87,"threads_started":1}
I20260812 16:35:04.815439 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:04.815124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:36345)
0812 16:35:04.815319 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:04.815388 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":99,"threads_started":1}
I20260812 16:35:04.950742 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:04.950491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:49693)
0812 16:35:04.950644 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:04.950692 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":69,"threads_started":1}
I20260812 16:35:05.474862 5368 raft_consensus.cc:482] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:35:05.474968 5368 raft_consensus.cc:504] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:35:05.475200 5368 leader_election.cc:288] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1466609a58564979844b45740b1b65ca (127.19.238.66:41157), caa889dc6e294f9fa9b6df0fb0c1d1e7 (127.19.238.67:37851)
W20260812 16:35:05.475642 4893 leader_election.cc:341] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1466609a58564979844b45740b1b65ca (127.19.238.66:41157): Not found: Tablet not found: dc0f1c03be6e4dcab7e6cb32b7724e5e
W20260812 16:35:05.475741 4894 leader_election.cc:341] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer caa889dc6e294f9fa9b6df0fb0c1d1e7 (127.19.238.67:37851): Not found: Tablet not found: dc0f1c03be6e4dcab7e6cb32b7724e5e
I20260812 16:35:05.475787 4894 leader_election.cc:302] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 37ca7c03644b497d8cdc2fcaa310218b; no voters: 1466609a58564979844b45740b1b65ca, caa889dc6e294f9fa9b6df0fb0c1d1e7
I20260812 16:35:05.475894 5368 raft_consensus.cc:2726] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:35:05.737972 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:05.737700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:56505)
0812 16:35:05.737856 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:05.737920 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":69,"threads_started":1}
I20260812 16:35:05.816439 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:05.816198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:41717)
0812 16:35:05.816347 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:05.816389 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260812 16:35:05.951658 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:05.951365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:55647)
0812 16:35:05.951564 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:05.951609 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":94,"threads_started":1}
I20260812 16:35:06.738907 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:06.738639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:59375)
0812 16:35:06.738801 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:06.738855 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":69,"threads_started":1}
I20260812 16:35:06.817351 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:06.817058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:51083)
0812 16:35:06.817222 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:06.817285 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":73,"threads_started":1}
I20260812 16:35:06.952613 5375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:06.952259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:34551)
0812 16:35:06.952505 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:06.952553 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":79,"threads_started":1}
I20260812 16:35:07.739768 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:07.739529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:40529)
0812 16:35:07.739670 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:07.739718 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260812 16:35:07.818311 5377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:07.818012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:59577)
0812 16:35:07.818181 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:07.818246 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":77,"threads_started":1}
I20260812 16:35:07.953519 5378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:07.953237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:34939)
0812 16:35:07.953393 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:07.953468 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":67,"threads_started":1}
I20260812 16:35:08.520537 5379 raft_consensus.cc:482] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:35:08.520681 5379 raft_consensus.cc:504] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1466609a58564979844b45740b1b65ca" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41157 } } peers { permanent_uuid: "caa889dc6e294f9fa9b6df0fb0c1d1e7" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 37851 } } peers { permanent_uuid: "37ca7c03644b497d8cdc2fcaa310218b" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 45577 } }
I20260812 16:35:08.520946 5379 leader_election.cc:288] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1466609a58564979844b45740b1b65ca (127.19.238.66:41157), caa889dc6e294f9fa9b6df0fb0c1d1e7 (127.19.238.67:37851)
W20260812 16:35:08.521401 4894 leader_election.cc:341] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer caa889dc6e294f9fa9b6df0fb0c1d1e7 (127.19.238.67:37851): Not found: Tablet not found: dc0f1c03be6e4dcab7e6cb32b7724e5e
W20260812 16:35:08.521492 4893 leader_election.cc:341] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1466609a58564979844b45740b1b65ca (127.19.238.66:41157): Not found: Tablet not found: dc0f1c03be6e4dcab7e6cb32b7724e5e
I20260812 16:35:08.521586 4893 leader_election.cc:302] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 37ca7c03644b497d8cdc2fcaa310218b; no voters: 1466609a58564979844b45740b1b65ca, caa889dc6e294f9fa9b6df0fb0c1d1e7
I20260812 16:35:08.521672 5379 raft_consensus.cc:2726] T dc0f1c03be6e4dcab7e6cb32b7724e5e P 37ca7c03644b497d8cdc2fcaa310218b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:35:08.740705 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:08.740446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:36589)
0812 16:35:08.740602 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:08.740649 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":68,"threads_started":1}
I20260812 16:35:08.819327 5383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:08.818999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:49457)
0812 16:35:08.819221 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:08.819273 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":99,"threads_started":1}
I20260812 16:35:08.954365 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:08.954126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:47673)
0812 16:35:08.954270 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:08.954316 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":68,"threads_started":1}
I20260812 16:35:09.741596 5387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:09.741353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:55169)
0812 16:35:09.741491 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:09.741533 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
I20260812 16:35:09.820315 5389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:09.820047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:40235)
0812 16:35:09.820216 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:09.820265 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":76,"threads_started":1}
I20260812 16:35:09.955085 5390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:09.954861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:46867)
0812 16:35:09.954995 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:09.955033 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260812 16:35:10.742439 5393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:10.742189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.65:37667)
0812 16:35:10.742352 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:10.742390 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260812 16:35:10.821346 5394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:10.821025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.66:56381)
0812 16:35:10.821247 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:10.821296 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":90,"threads_started":1}
I20260812 16:35:10.942395 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 4837
I20260812 16:35:10.950663 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 5027
I20260812 16:35:10.956830 5395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:10.956071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46389 (local address 127.19.238.67:58271)
0812 16:35:10.956720 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:10.956765 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":61,"threads_started":1}
I20260812 16:35:10.960744 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 5175
2026-08-12T16:35:10Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18493 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:35:11 dist-test-slave-mvvj krb5kdc[5398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:35:11 dist-test-slave-mvvj krb5kdc[5398](info): set up 2 sockets
Aug 12 16:35:11 dist-test-slave-mvvj krb5kdc[5398](info): commencing operation
krb5kdc: starting...
W20260812 16:35:12.089744 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:35:12 dist-test-slave-mvvj krb5kdc[5398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:35:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:35:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:35:12.228250 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:39511
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:39511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:35:12.317420 5415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:35:12.317670 5415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:35:12.317694 5415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:35:12.318813 5415 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 16:35:12.318849 5415 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 16:35:12.318862 5415 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 16:35:12.318873 5415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:35:12.318886 5415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:35:12.318899 5415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:35:12.318934 5415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:35:12.318948 5415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:35:12.320461 5415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:39511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:39511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:35:12.320683 5415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:35:12.320858 5415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 16:35:12.326182 5415 server_base.cc:1034] running on GCE node
W20260812 16:35:12.326156 5421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:35:12.326143 5423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:35:12.326149 5420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:35:12.326678 5415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:35:12.327035 5415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:35:12.328202 5415 hybrid_clock.cc:648] HybridClock initialized: now 1786552512328207 us; error 47 us; skew 500 ppm
Aug 12 16:35:12 dist-test-slave-mvvj krb5kdc[5398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552512, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:35:12.330710 5415 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:35:12.331231 5415 webserver.cc:460] Webserver started at http://127.19.238.126:33421/ using document root <none> and password file <none>
I20260812 16:35:12.331470 5415 fs_manager.cc:359] Metadata directory not provided
I20260812 16:35:12.331543 5415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:35:12.331705 5415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:35:12.332852 5415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "789ea6d5a384a32b17b6dabdffaa5241"
server_key_iv: "767d5771c849a281f9c01dd4dbff285a"
server_key_version: "encryptionkey@0"
I20260812 16:35:12.333258 5415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "789ea6d5a384a32b17b6dabdffaa5241"
server_key_iv: "767d5771c849a281f9c01dd4dbff285a"
server_key_version: "encryptionkey@0"
I20260812 16:35:12.334817 5415 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:35:12.335680 5430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:12.335918 5415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:35:12.336040 5415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "789ea6d5a384a32b17b6dabdffaa5241"
server_key_iv: "767d5771c849a281f9c01dd4dbff285a"
server_key_version: "encryptionkey@0"
I20260812 16:35:12.336120 5415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:35:12.349151 5415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:35:12.351296 5415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:35:12.351472 5415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:35:12.356333 5415 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:39511
I20260812 16:35:12.356369 5492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:39511 every 8 connection(s)
I20260812 16:35:12.356738 5415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:35:12.357628 5493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:35:12.360051 5493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: Bootstrap starting.
I20260812 16:35:12.360641 5493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: Neither blocks nor log segments found. Creating new log.
I20260812 16:35:12.360899 5493 log.cc:824] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:35:12.361588 5493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: No bootstrap required, opened a new log
I20260812 16:35:12.363204 5493 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39511 } }
I20260812 16:35:12.363358 5493 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:35:12.363381 5493 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f0f7f77b7134f0ba41c13b0f75c2e4e, State: Initialized, Role: FOLLOWER
I20260812 16:35:12.363499 5493 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39511 } }
I20260812 16:35:12.363607 5493 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:35:12.363653 5493 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:35:12.363705 5493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:35:12.364377 5493 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39511 } }
I20260812 16:35:12.364514 5493 leader_election.cc:302] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f0f7f77b7134f0ba41c13b0f75c2e4e; no voters:
I20260812 16:35:12.364724 5493 leader_election.cc:288] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:35:12.364755 5496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:35:12.364753 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 5415
I20260812 16:35:12.364874 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:35:12.364882 5496 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 1 LEADER]: Becoming Leader. State: Replica: 4f0f7f77b7134f0ba41c13b0f75c2e4e, State: Running, Role: LEADER
I20260812 16:35:12.365012 5493 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:35:12.365118 20409 external_mini_cluster.cc:1428] Setting key 52b48cff89ae89013d9cf097d580786b
I20260812 16:35:12.364959 5496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39511 } }
I20260812 16:35:12.365765 5496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f0f7f77b7134f0ba41c13b0f75c2e4e. Latest consensus state: current_term: 1 leader_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39511 } } }
I20260812 16:35:12.365957 5496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [sys.catalog]: This master's current role is: LEADER
I20260812 16:35:12.366026 5497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39511 } } }
I20260812 16:35:12.366211 5497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [sys.catalog]: This master's current role is: LEADER
I20260812 16:35:12.366271 5509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:35:12.366871 5509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:35:12.368777 5509 catalog_manager.cc:1347] Generated new cluster ID: 63c6237c128b4531ade0c866dc5058f2
I20260812 16:35:12.368839 5509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:35:12 dist-test-slave-mvvj krb5kdc[5398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:35:12.377007 5505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:12.365664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46032 (local address 127.19.238.126:39511)
0812 16:35:12.366039 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:12.366050 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:12.366090 (+ 40us) server_negotiation.cc:407] Connection header received
0812 16:35:12.366227 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:12.366235 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:12.366307 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:12.366454 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:35:12.367302 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.368778 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.369665 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.370090 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.370218 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:12.372823 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:35:12.372828 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:35:12.372829 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:35:12.372845 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:35:12.375219 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:12.375641 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:12.375647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:12.375648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:12.375758 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:12.376033 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:12.376034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:12.376034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:12.376316 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:35:12.376485 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:12.376614 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 16:35:12.376775 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":88,"threads_started":1}
I20260812 16:35:12.395449 5509 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:35:12.396261 5509 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:35:12.402562 5509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: Generated new TSK 0
I20260812 16:35:12.402828 5509 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
I20260812 16:35:12.443753 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:39511
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:35:12.534457 5523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:35:12.534713 5523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:35:12.534745 5523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:35:12.536247 5523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:35:12.536314 5523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:35:12.536348 5523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:35:12.538165 5523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:39511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:35:12.538550 5523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:35:12.538866 5523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:35:12.544886 5532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:35:12.544847 5529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:35:12.544865 5530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:35:12.545172 5523 server_base.cc:1034] running on GCE node
I20260812 16:35:12.545431 5523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:35:12.545707 5523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:35:12.546864 5523 hybrid_clock.cc:648] HybridClock initialized: now 1786552512546852 us; error 37 us; skew 500 ppm
Aug 12 16:35:12 dist-test-slave-mvvj krb5kdc[5398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552512, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:35:12.549962 5523 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:35:12.550559 5523 webserver.cc:460] Webserver started at http://127.19.238.65:34403/ using document root <none> and password file <none>
I20260812 16:35:12.550824 5523 fs_manager.cc:359] Metadata directory not provided
I20260812 16:35:12.550872 5523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:35:12.551059 5523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:35:12.552336 5523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "3b697ba16edf499e8812522375210c09"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "990352e8cb25fa958bfdfa823398fc91"
server_key_iv: "469226635dfbe525da629d23e6cc2532"
server_key_version: "encryptionkey@0"
I20260812 16:35:12.552728 5523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "3b697ba16edf499e8812522375210c09"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "990352e8cb25fa958bfdfa823398fc91"
server_key_iv: "469226635dfbe525da629d23e6cc2532"
server_key_version: "encryptionkey@0"
I20260812 16:35:12.554248 5523 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:35:12.555083 5541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:12.555330 5523 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:35:12.555459 5523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "3b697ba16edf499e8812522375210c09"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "990352e8cb25fa958bfdfa823398fc91"
server_key_iv: "469226635dfbe525da629d23e6cc2532"
server_key_version: "encryptionkey@0"
I20260812 16:35:12.555547 5523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:35:12.577597 5523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:35:12.579988 5523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:35:12.580170 5523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:35:12.580452 5523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:35:12.580881 5523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:35:12.580941 5523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:12.580987 5523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:35:12.581017 5523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:12.587205 5523 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:39495
I20260812 16:35:12.587384 5673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:39495 every 8 connection(s)
I20260812 16:35:12.587666 5523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:35:12.590962 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 5523
I20260812 16:35:12.591131 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:35:12.591351 20409 external_mini_cluster.cc:1428] Setting key b32978c2e10fd0bfa1d7d0a819b2d6bb
Aug 12 16:35:12 dist-test-slave-mvvj krb5kdc[5398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552512, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:35:12.598712 5505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:12.588553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:42055 (local address 127.19.238.126:39511)
0812 16:35:12.588630 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:12.588633 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:12.588877 (+ 244us) server_negotiation.cc:407] Connection header received
0812 16:35:12.589414 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:12.589416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:12.589426 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:12.589480 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:35:12.590449 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.590987 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.592534 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.592764 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.592849 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:12.595519 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:35:12.595522 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:35:12.595522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:35:12.595531 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:35:12.597119 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:12.597603 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:12.597604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:12.597605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:12.597661 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:12.597945 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:12.597946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:12.597946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:12.598081 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:35:12.598161 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:12.598478 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 16:35:12.598615 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 16:35:12.598891 5676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:12.588519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:42055)
0812 16:35:12.588742 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:12.588780 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:35:12.589288 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:35:12.589545 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:35:12.589548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:35:12.590150 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:35:12.590347 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:12.590352 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.591113 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.591114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:12.592429 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:12.592431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.592863 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.592864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:12.593010 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:12.593620 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:35:12.593629 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:35:12.595409 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:35:12.597217 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:35:12.597226 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:35:12.597229 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:35:12.597485 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:35:12.597745 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:35:12.597745 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:35:12.597746 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:35:12.597869 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:35:12.598151 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:35:12.598158 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:35:12.598324 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 16:35:12.598441 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 16:35:12.598623 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":131,"threads_started":1}
I20260812 16:35:12.599454 5674 heartbeater.cc:347] Connected to a master server at 127.19.238.126:39511
I20260812 16:35:12.599586 5674 heartbeater.cc:464] Registering TS with master...
I20260812 16:35:12.599814 5674 heartbeater.cc:511] Master 127.19.238.126:39511 requested a full tablet report, sending...
I20260812 16:35:12.600412 5451 ts_manager.cc:194] Registered new tserver with Master: 3b697ba16edf499e8812522375210c09 (127.19.238.65:39495)
WARNING: no policy specified for kudu/127.19.238.66@KRBTEST.COM; defaulting to no policy
I20260812 16:35:12.601089 5451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:42055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
I20260812 16:35:12.624833 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:39511
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:35:12.710960 5681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:35:12.711273 5681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:35:12.711323 5681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:35:12.712708 5681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:35:12.712800 5681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:35:12.712846 5681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:35:12.714570 5681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:39511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:35:12.714895 5681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:35:12.715171 5681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:35:12.721015 5687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:35:12.721069 5686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:35:12.721117 5681 server_base.cc:1034] running on GCE node
W20260812 16:35:12.721033 5689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:35:12.721446 5681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:35:12.721750 5681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:35:12.722927 5681 hybrid_clock.cc:648] HybridClock initialized: now 1786552512722907 us; error 45 us; skew 500 ppm
Aug 12 16:35:12 dist-test-slave-mvvj krb5kdc[5398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552512, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:35:12.725613 5681 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:35:12.726096 5681 webserver.cc:460] Webserver started at http://127.19.238.66:34795/ using document root <none> and password file <none>
I20260812 16:35:12.726336 5681 fs_manager.cc:359] Metadata directory not provided
I20260812 16:35:12.726411 5681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:35:12.726580 5681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:35:12.727885 5681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "769309b27347411c0b3e6b009bbe143b"
server_key_iv: "5018ef98243e64f478b5bae3c1490930"
server_key_version: "encryptionkey@0"
I20260812 16:35:12.728300 5681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "769309b27347411c0b3e6b009bbe143b"
server_key_iv: "5018ef98243e64f478b5bae3c1490930"
server_key_version: "encryptionkey@0"
I20260812 16:35:12.729786 5681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:35:12.730803 5698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:12.731042 5681 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:35:12.731169 5681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "769309b27347411c0b3e6b009bbe143b"
server_key_iv: "5018ef98243e64f478b5bae3c1490930"
server_key_version: "encryptionkey@0"
I20260812 16:35:12.731262 5681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:35:12.745100 5681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:35:12.748005 5681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:35:12.748211 5681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:35:12.748485 5681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:35:12.748868 5681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:35:12.748926 5681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:12.748970 5681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:35:12.749002 5681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:12.755548 5681 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:37659
I20260812 16:35:12.755635 5826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:37659 every 8 connection(s)
I20260812 16:35:12.756059 5681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:35:12.761162 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 5681
I20260812 16:35:12.761312 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:35:12.761543 20409 external_mini_cluster.cc:1428] Setting key 5cb92398596d6b362114412ab1943e11
Aug 12 16:35:12 dist-test-slave-mvvj krb5kdc[5398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552512, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:35:12.769556 5505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:12.756938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:38261 (local address 127.19.238.126:39511)
0812 16:35:12.757017 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:12.757019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:12.757243 (+ 224us) server_negotiation.cc:407] Connection header received
0812 16:35:12.757874 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:12.757875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:12.757887 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:12.757961 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:35:12.759161 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.759697 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.760943 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.761216 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.761391 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:12.765018 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:35:12.765020 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:35:12.765020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:35:12.765028 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:35:12.767180 (+ 2152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:12.767958 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:12.767966 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:12.767966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:12.768057 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:12.768481 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:12.768482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:12.768483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:12.768656 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:35:12.768779 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:12.769266 (+ 487us) server_negotiation.cc:299] Negotiation successful
0812 16:35:12.769406 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 16:35:12.769661 5831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:12.756895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:38261)
0812 16:35:12.757101 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:12.757143 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:35:12.757724 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:35:12.758055 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:35:12.758059 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:35:12.758748 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:35:12.759057 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:12.759063 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.759813 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.759814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:12.760816 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:12.760818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.761399 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.761400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:12.761584 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:12.762375 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:35:12.762387 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:35:12.764850 (+ 2463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:35:12.767338 (+ 2488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:35:12.767351 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:35:12.767354 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:35:12.767779 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:35:12.768169 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:35:12.768170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:35:12.768170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:35:12.768362 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:35:12.768759 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:35:12.768767 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:35:12.769020 (+ 253us) client_negotiation.cc:769] Sending connection context
0812 16:35:12.769201 (+ 181us) client_negotiation.cc:240] Negotiation successful
0812 16:35:12.769399 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":110,"threads_started":1}
I20260812 16:35:12.770491 5827 heartbeater.cc:347] Connected to a master server at 127.19.238.126:39511
I20260812 16:35:12.770648 5827 heartbeater.cc:464] Registering TS with master...
I20260812 16:35:12.770891 5827 heartbeater.cc:511] Master 127.19.238.126:39511 requested a full tablet report, sending...
I20260812 16:35:12.771570 5451 ts_manager.cc:194] Registered new tserver with Master: 15cb6a42b7994dc6ae21bd1d0d3062bb (127.19.238.66:37659)
WARNING: no policy specified for kudu/127.19.238.67@KRBTEST.COM; defaulting to no policy
I20260812 16:35:12.772060 5451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:38261
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
I20260812 16:35:12.797143 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:39511
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:35:12.890462 5837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:35:12.890687 5837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:35:12.890713 5837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:35:12.891987 5837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:35:12.892037 5837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:35:12.892068 5837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:35:12.893699 5837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:39511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:35:12.893971 5837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:35:12.894245 5837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:35:12.899758 5846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:35:12.899915 5844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:35:12.899879 5843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:35:12.899959 5837 server_base.cc:1034] running on GCE node
I20260812 16:35:12.900401 5837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:35:12.900689 5837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:35:12.901839 5837 hybrid_clock.cc:648] HybridClock initialized: now 1786552512901827 us; error 34 us; skew 500 ppm
Aug 12 16:35:12 dist-test-slave-mvvj krb5kdc[5398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552512, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:35:12.911027 5837 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:35:12.911612 5837 webserver.cc:460] Webserver started at http://127.19.238.67:45187/ using document root <none> and password file <none>
I20260812 16:35:12.911933 5837 fs_manager.cc:359] Metadata directory not provided
I20260812 16:35:12.912011 5837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:35:12.912182 5837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:35:12.913341 5837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "3f9e8fd85db0448ab43452f87bb4c0e1"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "75c1d9d46c2accbbe4e27eeb3dd4f52f"
server_key_iv: "38222d3ea6ebdcf9f06893d72f243298"
server_key_version: "encryptionkey@0"
I20260812 16:35:12.913762 5837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "3f9e8fd85db0448ab43452f87bb4c0e1"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "75c1d9d46c2accbbe4e27eeb3dd4f52f"
server_key_iv: "38222d3ea6ebdcf9f06893d72f243298"
server_key_version: "encryptionkey@0"
I20260812 16:35:12.915197 5837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:35:12.916105 5854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:12.916285 5837 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:35:12.916397 5837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "3f9e8fd85db0448ab43452f87bb4c0e1"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "75c1d9d46c2accbbe4e27eeb3dd4f52f"
server_key_iv: "38222d3ea6ebdcf9f06893d72f243298"
server_key_version: "encryptionkey@0"
I20260812 16:35:12.916527 5837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:35:12.928043 5837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:35:12.930284 5837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:35:12.930476 5837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:35:12.930737 5837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:35:12.931083 5837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:35:12.931141 5837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:12.931190 5837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:35:12.931214 5837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:12.937952 5837 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:39859
I20260812 16:35:12.937994 5988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:39859 every 8 connection(s)
I20260812 16:35:12.938390 5837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:35:12.944135 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 5837
I20260812 16:35:12.944247 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:35:12.944453 20409 external_mini_cluster.cc:1428] Setting key 5febf3fe4600e691cec854c117fedf05
Aug 12 16:35:12 dist-test-slave-mvvj krb5kdc[5398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552512, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:35:12.948786 5505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:12.939260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:47003 (local address 127.19.238.126:39511)
0812 16:35:12.939335 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:12.939337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:12.939586 (+ 249us) server_negotiation.cc:407] Connection header received
0812 16:35:12.940246 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:12.940247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:12.940261 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:12.940326 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:35:12.941343 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.941865 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.942985 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.943205 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.943293 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:12.945990 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:35:12.945991 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:35:12.945992 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:35:12.946000 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:35:12.947245 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:12.947689 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:12.947691 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:12.947691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:12.947757 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:12.948062 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:12.948063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:12.948064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:12.948192 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:35:12.948269 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:12.948597 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 16:35:12.948692 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 16:35:12.948989 5991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:12.939268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:47003)
0812 16:35:12.939457 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:12.939496 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:35:12.940126 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:35:12.940409 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:35:12.940412 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:35:12.941024 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:35:12.941230 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:12.941235 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.941980 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.941982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:12.942872 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:12.942875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.943307 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.943308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:12.943447 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:12.944076 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:35:12.944089 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:35:12.945872 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:35:12.947335 (+ 1463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:35:12.947344 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:35:12.947346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:35:12.947590 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:35:12.947858 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:35:12.947860 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:35:12.947860 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:35:12.947983 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:35:12.948260 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:35:12.948268 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:35:12.948449 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 16:35:12.948559 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 16:35:12.948735 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":88,"threads_started":1}
I20260812 16:35:12.949543 5989 heartbeater.cc:347] Connected to a master server at 127.19.238.126:39511
I20260812 16:35:12.949659 5989 heartbeater.cc:464] Registering TS with master...
I20260812 16:35:12.949851 5989 heartbeater.cc:511] Master 127.19.238.126:39511 requested a full tablet report, sending...
I20260812 16:35:12.950275 5451 ts_manager.cc:194] Registered new tserver with Master: 3f9e8fd85db0448ab43452f87bb4c0e1 (127.19.238.67:39859)
I20260812 16:35:12.950716 5451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:47003
I20260812 16:35:12.956491 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:35:12.964825 5505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:12.957907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46036 (local address 127.19.238.126:39511)
0812 16:35:12.958010 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:12.958012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:12.958116 (+ 104us) server_negotiation.cc:407] Connection header received
0812 16:35:12.958213 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:12.958215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:12.958226 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:12.958370 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:35:12.958970 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.959480 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.960424 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.960629 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.960702 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:12.962025 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:35:12.962027 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:35:12.962027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:35:12.962035 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:35:12.963492 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:12.963921 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:12.963922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:12.963923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:12.963985 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:12.964196 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:12.964197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:12.964198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:12.964376 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:35:12.964490 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:12.964624 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 16:35:12.964710 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 16:35:12.966861 5451 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46036:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:35:12.968680 5451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:35:12.975775 6001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:12.971328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:39859 (local address 127.0.0.1:43354)
0812 16:35:12.971657 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:12.971705 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 16:35:12.971883 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:35:12.972194 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:35:12.972202 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:35:12.972204 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:35:12.972263 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:12.972265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.973682 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.973683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:12.974684 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:12.974686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.975363 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.975365 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:12.975471 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:12.975483 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 16:35:12.975580 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 16:35:12.975685 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":53,"thread_start_us":58,"threads_started":1}
I20260812 16:35:12.975783 6005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:12.971572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43354 (local address 127.19.238.67:39859)
0812 16:35:12.971748 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:12.971752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:12.971764 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:35:12.972005 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:12.972008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:12.972024 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:12.972097 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:35:12.972348 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.973557 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.974775 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.975255 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.975352 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:12.975415 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:12.975602 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 16:35:12.975676 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260812 16:35:12.975970 6004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:12.971464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52198 (local address 127.19.238.65:39495)
0812 16:35:12.971637 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:12.971642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:12.971999 (+ 357us) server_negotiation.cc:407] Connection header received
0812 16:35:12.972126 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:12.972129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:12.972145 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:12.972197 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:35:12.972464 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.973570 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.974798 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.975509 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.975555 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:12.975606 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:12.975810 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 16:35:12.975891 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260812 16:35:12.976022 6006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:12.971646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60916 (local address 127.19.238.66:37659)
0812 16:35:12.971863 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:12.971868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:12.971881 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:35:12.971996 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:12.971998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:12.972040 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:12.972113 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:35:12.972362 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.973106 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.974514 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.975259 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.975352 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:12.975415 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:12.975881 (+ 466us) server_negotiation.cc:299] Negotiation successful
0812 16:35:12.975931 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":84,"threads_started":1}
I20260812 16:35:12.976238 6002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:12.971400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:39495 (local address 127.0.0.1:52198)
0812 16:35:12.971926 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:12.971936 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:35:12.972027 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:35:12.972309 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:35:12.972309 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:35:12.972311 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:35:12.972372 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:12.972373 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.973696 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.973697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:12.974687 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:12.974690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.975668 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.975669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:12.975719 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:12.975722 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:35:12.976110 (+ 388us) client_negotiation.cc:240] Negotiation successful
0812 16:35:12.976164 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":1,"thread_start_us":104,"threads_started":1}
I20260812 16:35:12.975800 6003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:12.971329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:37659 (local address 127.0.0.1:60916)
0812 16:35:12.971760 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:12.971772 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:35:12.971862 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:35:12.972198 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:35:12.972205 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:35:12.972207 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:35:12.972275 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:12.972277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.973234 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.973236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:12.974416 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:12.974418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:12.975455 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:12.975456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:12.975522 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:12.975528 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 16:35:12.975578 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 16:35:12.975670 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":4,"thread_start_us":85,"threads_started":1}
I20260812 16:35:12.977150 5913 tablet_service.cc:1467] Processing CreateTablet for tablet b1521454c65c4e3abc3b2f027aa1c214 (DEFAULT_TABLE table=test-table [id=6170805c4215465085d785d1f9e65f7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:35:12.977222 5597 tablet_service.cc:1467] Processing CreateTablet for tablet b1521454c65c4e3abc3b2f027aa1c214 (DEFAULT_TABLE table=test-table [id=6170805c4215465085d785d1f9e65f7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:35:12.977418 5754 tablet_service.cc:1467] Processing CreateTablet for tablet b1521454c65c4e3abc3b2f027aa1c214 (DEFAULT_TABLE table=test-table [id=6170805c4215465085d785d1f9e65f7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:35:12.977536 5913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1521454c65c4e3abc3b2f027aa1c214. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:35:12.977566 5597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1521454c65c4e3abc3b2f027aa1c214. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:35:12.977715 5754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1521454c65c4e3abc3b2f027aa1c214. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:35:12.981138 6008 tablet_bootstrap.cc:492] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09: Bootstrap starting.
I20260812 16:35:12.981577 6007 tablet_bootstrap.cc:492] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1: Bootstrap starting.
I20260812 16:35:12.981878 6008 tablet_bootstrap.cc:654] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09: Neither blocks nor log segments found. Creating new log.
I20260812 16:35:12.982231 6007 tablet_bootstrap.cc:654] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1: Neither blocks nor log segments found. Creating new log.
I20260812 16:35:12.982414 6008 log.cc:824] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09: Log is configured to *not* fsync() on all Append() calls
I20260812 16:35:12.982764 6009 tablet_bootstrap.cc:492] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb: Bootstrap starting.
I20260812 16:35:12.982858 6007 log.cc:824] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1: Log is configured to *not* fsync() on all Append() calls
I20260812 16:35:12.983347 6008 tablet_bootstrap.cc:492] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09: No bootstrap required, opened a new log
I20260812 16:35:12.983426 6008 ts_tablet_manager.cc:1397] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:35:12.983480 6009 tablet_bootstrap.cc:654] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb: Neither blocks nor log segments found. Creating new log.
I20260812 16:35:12.983639 6007 tablet_bootstrap.cc:492] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1: No bootstrap required, opened a new log
I20260812 16:35:12.983734 6007 ts_tablet_manager.cc:1397] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:35:12.984071 6009 log.cc:824] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb: Log is configured to *not* fsync() on all Append() calls
I20260812 16:35:12.985000 6009 tablet_bootstrap.cc:492] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb: No bootstrap required, opened a new log
I20260812 16:35:12.985109 6009 ts_tablet_manager.cc:1397] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:35:12.985606 6008 raft_consensus.cc:348] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:12.985787 6008 raft_consensus.cc:374] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:35:12.985852 6008 raft_consensus.cc:729] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b697ba16edf499e8812522375210c09, State: Initialized, Role: FOLLOWER
I20260812 16:35:12.985932 6007 raft_consensus.cc:348] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:12.986001 6008 consensus_queue.cc:260] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:12.986110 6007 raft_consensus.cc:374] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:35:12.986150 6007 raft_consensus.cc:729] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f9e8fd85db0448ab43452f87bb4c0e1, State: Initialized, Role: FOLLOWER
I20260812 16:35:12.986292 6007 consensus_queue.cc:260] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:12.986351 6008 ts_tablet_manager.cc:1428] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:35:12.986505 5674 heartbeater.cc:503] Master 127.19.238.126:39511 was elected leader, sending a full tablet report...
I20260812 16:35:12.986581 6007 ts_tablet_manager.cc:1428] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 16:35:12.986725 5989 heartbeater.cc:503] Master 127.19.238.126:39511 was elected leader, sending a full tablet report...
I20260812 16:35:12.987324 6009 raft_consensus.cc:348] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:12.987510 6009 raft_consensus.cc:374] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:35:12.987556 6009 raft_consensus.cc:729] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15cb6a42b7994dc6ae21bd1d0d3062bb, State: Initialized, Role: FOLLOWER
I20260812 16:35:12.987682 6009 consensus_queue.cc:260] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:12.987998 5827 heartbeater.cc:503] Master 127.19.238.126:39511 was elected leader, sending a full tablet report...
I20260812 16:35:12.988018 6009 ts_tablet_manager.cc:1428] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
W20260812 16:35:13.006772 5829 tablet.cc:2369] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:35:13.040354 6015 raft_consensus.cc:482] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:35:13.040488 6015 raft_consensus.cc:504] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:13.040885 6015 leader_election.cc:288] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f9e8fd85db0448ab43452f87bb4c0e1 (127.19.238.67:39859), 15cb6a42b7994dc6ae21bd1d0d3062bb (127.19.238.66:37659)
I20260812 16:35:13.044991 6006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:13.041188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:38985 (local address 127.19.238.66:37659)
0812 16:35:13.041245 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:13.041248 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:13.041348 (+ 100us) server_negotiation.cc:407] Connection header received
0812 16:35:13.041383 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:13.041384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:13.041396 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:13.041461 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:35:13.041628 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:13.042361 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:13.043933 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:13.044558 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:13.044627 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:13.044636 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:13.044874 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 16:35:13.044917 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 16:35:13.044991 6005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:13.041215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:50347 (local address 127.19.238.67:39859)
0812 16:35:13.041271 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:13.041273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:13.041288 (+ 15us) server_negotiation.cc:407] Connection header received
0812 16:35:13.041314 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:13.041315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:13.041338 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:13.041396 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:35:13.041620 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:13.042361 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:13.043938 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:13.044523 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:13.044613 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:13.044623 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:13.044877 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 16:35:13.044917 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 16:35:13.045010 5676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:13.041025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:39859 (local address 127.19.238.65:50347)
0812 16:35:13.041098 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:13.041109 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:35:13.041173 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:35:13.041430 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:35:13.041431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:35:13.041432 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:35:13.041513 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:13.041516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:13.042483 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:13.042484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:13.043818 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:13.043822 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:13.044663 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:13.044665 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:13.044777 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:13.044785 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 16:35:13.044857 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 16:35:13.044910 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":97}
I20260812 16:35:13.045017 6019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:13.041063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:37659 (local address 127.19.238.65:38985)
0812 16:35:13.041270 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:13.041286 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 16:35:13.041342 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:35:13.041498 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:35:13.041499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:35:13.041500 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:35:13.041557 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:13.041559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:13.042483 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:13.042484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:13.043818 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:13.043822 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:13.044663 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:13.044665 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:13.044777 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:13.044785 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 16:35:13.044857 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 16:35:13.044910 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":61,"thread_start_us":121,"threads_started":1}
I20260812 16:35:13.045452 5936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1521454c65c4e3abc3b2f027aa1c214" candidate_uuid: "3b697ba16edf499e8812522375210c09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" is_pre_election: true
I20260812 16:35:13.045670 5936 raft_consensus.cc:2454] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b697ba16edf499e8812522375210c09 in term 0.
I20260812 16:35:13.045902 5776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1521454c65c4e3abc3b2f027aa1c214" candidate_uuid: "3b697ba16edf499e8812522375210c09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" is_pre_election: true
I20260812 16:35:13.045945 5545 leader_election.cc:302] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b697ba16edf499e8812522375210c09, 3f9e8fd85db0448ab43452f87bb4c0e1; no voters:
I20260812 16:35:13.046082 5776 raft_consensus.cc:2454] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b697ba16edf499e8812522375210c09 in term 0.
I20260812 16:35:13.046105 6015 raft_consensus.cc:2781] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:35:13.046176 6015 raft_consensus.cc:482] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:35:13.046212 6015 raft_consensus.cc:3037] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:35:13.047122 6015 raft_consensus.cc:504] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:13.047289 6015 leader_election.cc:288] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 1 election: Requested vote from peers 3f9e8fd85db0448ab43452f87bb4c0e1 (127.19.238.67:39859), 15cb6a42b7994dc6ae21bd1d0d3062bb (127.19.238.66:37659)
I20260812 16:35:13.047444 5936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1521454c65c4e3abc3b2f027aa1c214" candidate_uuid: "3b697ba16edf499e8812522375210c09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1"
I20260812 16:35:13.047490 5776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1521454c65c4e3abc3b2f027aa1c214" candidate_uuid: "3b697ba16edf499e8812522375210c09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb"
I20260812 16:35:13.047540 5936 raft_consensus.cc:3037] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:35:13.047616 5776 raft_consensus.cc:3037] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:35:13.048343 5936 raft_consensus.cc:2454] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b697ba16edf499e8812522375210c09 in term 1.
I20260812 16:35:13.048517 5545 leader_election.cc:302] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b697ba16edf499e8812522375210c09, 3f9e8fd85db0448ab43452f87bb4c0e1; no voters:
I20260812 16:35:13.048547 5776 raft_consensus.cc:2454] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b697ba16edf499e8812522375210c09 in term 1.
I20260812 16:35:13.048620 6015 raft_consensus.cc:2781] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:35:13.048812 6015 raft_consensus.cc:686] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [term 1 LEADER]: Becoming Leader. State: Replica: 3b697ba16edf499e8812522375210c09, State: Running, Role: LEADER
I20260812 16:35:13.048960 6015 consensus_queue.cc:237] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:13.050012 5450 catalog_manager.cc:5495] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b697ba16edf499e8812522375210c09 (127.19.238.65). New cstate: current_term: 1 leader_uuid: "3b697ba16edf499e8812522375210c09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } health_report { overall_health: HEALTHY } } }
I20260812 16:35:13.070195 6004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:13.067872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52204 (local address 127.19.238.65:39495)
0812 16:35:13.067952 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:13.067954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:13.067962 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:35:13.068001 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:13.068002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:13.068022 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:13.068089 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:35:13.068219 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:13.068732 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:13.069467 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:13.069687 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:13.069753 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:13.069886 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:35:13.069989 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:35:13.070053 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:13.070095 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 16:35:13.070115 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 16:35:13.072176 5776 raft_consensus.cc:1261] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb [term 1 FOLLOWER]: Refusing update from remote peer 3b697ba16edf499e8812522375210c09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:35:13.072176 5936 raft_consensus.cc:1261] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1 [term 1 FOLLOWER]: Refusing update from remote peer 3b697ba16edf499e8812522375210c09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:35:13.072599 6020 consensus_queue.cc:1035] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:35:13.072728 6015 consensus_queue.cc:1035] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:35:13.075124 6026 mvcc.cc:205] Tried to move back new op lower bound from 7317719093540646912 to 7317719093449261056. Current Snapshot: MvccSnapshot[applied={T|T < 7317719093540646912}]
I20260812 16:35:13.078145 5450 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:35:13.078269 5450 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:35:13.079114 5450 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: Sending DeleteTablet for 3 replicas of tablet b1521454c65c4e3abc3b2f027aa1c214
I20260812 16:35:13.079443 5913 tablet_service.cc:1514] Processing DeleteTablet for tablet b1521454c65c4e3abc3b2f027aa1c214 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:35:13 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:43354
I20260812 16:35:13.079483 5597 tablet_service.cc:1514] Processing DeleteTablet for tablet b1521454c65c4e3abc3b2f027aa1c214 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:35:13 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:52198
I20260812 16:35:13.079512 5754 tablet_service.cc:1514] Processing DeleteTablet for tablet b1521454c65c4e3abc3b2f027aa1c214 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:35:13 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:60916
I20260812 16:35:13.079741 6039 tablet_replica.cc:330] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09: stopping tablet replica
I20260812 16:35:13.079749 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 5523
I20260812 16:35:13.081799 6040 tablet_replica.cc:330] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb: stopping tablet replica
I20260812 16:35:13.081939 6040 raft_consensus.cc:2229] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:35:13.082072 6040 raft_consensus.cc:2258] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:35:13.082588 6040 ts_tablet_manager.cc:1905] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:35:13.083577 6038 tablet_replica.cc:330] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1: stopping tablet replica
I20260812 16:35:13.083707 6038 raft_consensus.cc:2229] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:35:13.084059 6038 raft_consensus.cc:2258] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:35:13.084391 6040 ts_tablet_manager.cc:1918] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:35:13.084663 6038 ts_tablet_manager.cc:1905] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:35:13.084757 6040 log.cc:1196] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/b1521454c65c4e3abc3b2f027aa1c214
I20260812 16:35:13.085173 6040 ts_tablet_manager.cc:1939] T b1521454c65c4e3abc3b2f027aa1c214 P 15cb6a42b7994dc6ae21bd1d0d3062bb: Deleting consensus metadata
I20260812 16:35:13.085729 5431 catalog_manager.cc:4841] TS 15cb6a42b7994dc6ae21bd1d0d3062bb (127.19.238.66:37659): tablet b1521454c65c4e3abc3b2f027aa1c214 (table test-table [id=6170805c4215465085d785d1f9e65f7e]) successfully deleted
I20260812 16:35:13.086380 6038 ts_tablet_manager.cc:1918] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:35:13.086469 6038 log.cc:1196] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/b1521454c65c4e3abc3b2f027aa1c214
I20260812 16:35:13.086745 6038 ts_tablet_manager.cc:1939] T b1521454c65c4e3abc3b2f027aa1c214 P 3f9e8fd85db0448ab43452f87bb4c0e1: Deleting consensus metadata
I20260812 16:35:13.087319 5435 catalog_manager.cc:4841] TS 3f9e8fd85db0448ab43452f87bb4c0e1 (127.19.238.67:39859): tablet b1521454c65c4e3abc3b2f027aa1c214 (table test-table [id=6170805c4215465085d785d1f9e65f7e]) successfully deleted
W20260812 16:35:13.089325 5434 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.238.65:39495 (error 108)
I20260812 16:35:13.089633 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 5681
I20260812 16:35:13.096465 6003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:13.096266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:39495 (local address 127.0.0.1:52206)
0812 16:35:13.096342 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:13.096411 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:39495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 16:35:13.096671 5434 catalog_manager.cc:4584] TS 3b697ba16edf499e8812522375210c09 (127.19.238.65:39495): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b1521454c65c4e3abc3b2f027aa1c214: Network error: Client connection negotiation failed: client connection to 127.19.238.65:39495: connect: Connection refused (error 111)
I20260812 16:35:13.097213 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 5837
I20260812 16:35:13.103966 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 5415
I20260812 16:35:33.112715 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:39511
--webserver_interface=127.19.238.126
--webserver_port=33421
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:39511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:35:33.198916 6042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:35:33.199172 6042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:35:33.199213 6042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:35:33.200424 6042 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 16:35:33.200491 6042 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 16:35:33.200512 6042 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 16:35:33.200531 6042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:35:33.200560 6042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:35:33.200588 6042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:35:33.200613 6042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:35:33.200639 6042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:35:33.202587 6042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:39511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:39511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=33421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:35:33.202893 6042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:35:33.203137 6042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:35:33.213645 6051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:35:33.213645 6048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:35:33.213721 6042 server_base.cc:1034] running on GCE node
W20260812 16:35:33.213737 6049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:35:33.214201 6042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:35:33.214527 6042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:35:33.215692 6042 hybrid_clock.cc:648] HybridClock initialized: now 1786552533215682 us; error 33 us; skew 500 ppm
Aug 12 16:35:33 dist-test-slave-mvvj krb5kdc[5398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552533, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:35:33.223594 6042 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:35:33.224239 6042 webserver.cc:460] Webserver started at http://127.19.238.126:33421/ using document root <none> and password file <none>
I20260812 16:35:33.224514 6042 fs_manager.cc:359] Metadata directory not provided
I20260812 16:35:33.224608 6042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:35:33.226233 6042 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:35:33.227087 6058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:33.227353 6042 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:35:33.227523 6042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "789ea6d5a384a32b17b6dabdffaa5241"
server_key_iv: "767d5771c849a281f9c01dd4dbff285a"
server_key_version: "encryptionkey@0"
I20260812 16:35:33.228066 6042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:35:33.254454 6042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:35:33.256831 6042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:35:33.257006 6042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:35:33.261523 6042 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:39511
I20260812 16:35:33.261562 6115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:39511 every 8 connection(s)
I20260812 16:35:33.261992 6042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:35:33.263587 6116 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:35:33.264369 6116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: Bootstrap starting.
I20260812 16:35:33.269683 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 6042
I20260812 16:35:33.270037 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:39495
--local_ip_for_outbound_sockets=127.19.238.65
--tserver_master_addrs=127.19.238.126:39511
--webserver_port=34403
--webserver_interface=127.19.238.65
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:35:33.273272 6116 log.cc:824] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:35:33.275389 6116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:35:33.275729 6116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: Bootstrap complete.
I20260812 16:35:33.278121 6116 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39511 } }
I20260812 16:35:33.278381 6116 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f0f7f77b7134f0ba41c13b0f75c2e4e, State: Initialized, Role: FOLLOWER
I20260812 16:35:33.278548 6116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39511 } }
I20260812 16:35:33.278636 6116 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:35:33.278698 6116 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:35:33.278766 6116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 1 FOLLOWER]: Advancing to term 2
I20260812 16:35:33.279935 6116 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39511 } }
I20260812 16:35:33.280049 6116 leader_election.cc:302] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f0f7f77b7134f0ba41c13b0f75c2e4e; no voters:
I20260812 16:35:33.280267 6116 leader_election.cc:288] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 16:35:33.280304 6121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 2 FOLLOWER]: Leader election won for term 2
I20260812 16:35:33.280423 6121 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 2 LEADER]: Becoming Leader. State: Replica: 4f0f7f77b7134f0ba41c13b0f75c2e4e, State: Running, Role: LEADER
I20260812 16:35:33.280558 6116 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:35:33.280586 6121 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39511 } }
I20260812 16:35:33.280946 6122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39511 } } }
I20260812 16:35:33.280966 6123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f0f7f77b7134f0ba41c13b0f75c2e4e. Latest consensus state: current_term: 2 leader_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39511 } } }
I20260812 16:35:33.281031 6122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [sys.catalog]: This master's current role is: LEADER
I20260812 16:35:33.281057 6123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [sys.catalog]: This master's current role is: LEADER
I20260812 16:35:33.282611 6140 catalog_manager.cc:1259] Loaded cluster ID: 63c6237c128b4531ade0c866dc5058f2
I20260812 16:35:33.282668 6140 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: loading cluster ID for follower catalog manager: success
I20260812 16:35:33.283823 6140 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: acquiring CA information for follower catalog manager: success
I20260812 16:35:33.284283 6140 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:35:33.284482 6130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:35:33.284993 6130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:35:33.285086 6130 catalog_manager.cc:1259] Loaded cluster ID: 63c6237c128b4531ade0c866dc5058f2
I20260812 16:35:33.285117 6130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:35:33.285290 6130 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:35:33.285353 6130 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: Loaded TSK: 0
I20260812 16:35:33.297640 6130 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: Generated new TSK 1
I20260812 16:35:33.297763 6130 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 16:35:33.360625 6118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:35:33.360837 6118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:35:33.360862 6118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:35:33.362044 6118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:35:33.362093 6118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:35:33.362121 6118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:35:33.364079 6118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:39495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=34403
--tserver_master_addrs=127.19.238.126:39511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:35:33.364347 6118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:35:33.364650 6118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:35:33.369959 6149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:35:33.369995 6150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:35:33.370275 6118 server_base.cc:1034] running on GCE node
W20260812 16:35:33.369973 6153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:35:33.370653 6118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:35:33.370922 6118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:35:33.372116 6118 hybrid_clock.cc:648] HybridClock initialized: now 1786552533372091 us; error 40 us; skew 500 ppm
Aug 12 16:35:33 dist-test-slave-mvvj krb5kdc[5398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552533, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:35:33.374966 6118 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:35:33.375579 6118 webserver.cc:460] Webserver started at http://127.19.238.65:34403/ using document root <none> and password file <none>
I20260812 16:35:33.375897 6118 fs_manager.cc:359] Metadata directory not provided
I20260812 16:35:33.375994 6118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:35:33.377502 6118 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:35:33.378279 6162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:33.378510 6118 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:35:33.378614 6118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "3b697ba16edf499e8812522375210c09"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "990352e8cb25fa958bfdfa823398fc91"
server_key_iv: "469226635dfbe525da629d23e6cc2532"
server_key_version: "encryptionkey@0"
I20260812 16:35:33.379113 6118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:35:33.401393 6118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:35:33.403964 6118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:35:33.404138 6118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:35:33.404392 6118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:35:33.404893 6170 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 16:35:33.406868 6118 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 16:35:33.406955 6118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 16:35:33.407006 6118 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 16:35:33.407675 6118 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 16:35:33.407733 6118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 16:35:33.407795 6170 tablet_bootstrap.cc:492] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09: Bootstrap starting.
I20260812 16:35:33.416723 6118 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:39495
I20260812 16:35:33.417181 6118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:35:33.419556 6170 log.cc:824] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09: Log is configured to *not* fsync() on all Append() calls
I20260812 16:35:33.420635 6293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:39495 every 8 connection(s)
I20260812 16:35:33.422626 6170 tablet_bootstrap.cc:492] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:35:33.422974 6170 tablet_bootstrap.cc:492] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09: Bootstrap complete.
I20260812 16:35:33.424335 6170 ts_tablet_manager.cc:1397] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.013s
I20260812 16:35:33.425772 6170 raft_consensus.cc:348] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:33.426028 6170 raft_consensus.cc:729] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b697ba16edf499e8812522375210c09, State: Initialized, Role: FOLLOWER
I20260812 16:35:33.426170 6170 consensus_queue.cc:260] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:33.426491 6170 ts_tablet_manager.cc:1428] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 16:35:33.426739 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 6118
I20260812 16:35:33.427088 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:37659
--local_ip_for_outbound_sockets=127.19.238.66
--tserver_master_addrs=127.19.238.126:39511
--webserver_port=34795
--webserver_interface=127.19.238.66
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:35:33 dist-test-slave-mvvj krb5kdc[5398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552533, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:35:33.436483 6297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.421319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:46147)
0812 16:35:33.424003 (+ 2684us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:33.424044 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:35:33.424645 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:35:33.425074 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:35:33.425079 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:35:33.425372 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:35:33.425589 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:33.425594 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.427056 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.427057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:33.428058 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:33.428061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.428913 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.428915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:33.429100 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.429896 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:35:33.429909 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:35:33.431918 (+ 2009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:35:33.434549 (+ 2631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:35:33.434557 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:35:33.434558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:35:33.434824 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:35:33.435197 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:35:33.435198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:35:33.435198 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:35:33.435343 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:35:33.435955 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:35:33.435962 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:35:33.436089 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 16:35:33.436177 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 16:35:33.436317 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2634,"thread_start_us":109,"threads_started":1}
I20260812 16:35:33.436728 6298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.421383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:46147 (local address 127.19.238.126:39511)
0812 16:35:33.424648 (+ 3265us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:33.424658 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:33.424708 (+ 50us) server_negotiation.cc:407] Connection header received
0812 16:35:33.424846 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:33.424851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:33.424895 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:33.425031 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:35:33.425711 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.426912 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.428149 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.428798 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.428926 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.432068 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:35:33.432073 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:35:33.432074 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:35:33.432090 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:35:33.434436 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:33.434937 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:33.434941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:33.434942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:33.435091 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:33.435436 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:33.435437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:33.435438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:33.435817 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:35:33.435953 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:33.436291 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 16:35:33.436473 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3201,"thread_start_us":85,"threads_started":1}
I20260812 16:35:33.437462 6294 heartbeater.cc:347] Connected to a master server at 127.19.238.126:39511
I20260812 16:35:33.437595 6294 heartbeater.cc:464] Registering TS with master...
I20260812 16:35:33.437821 6294 heartbeater.cc:511] Master 127.19.238.126:39511 requested a full tablet report, sending...
I20260812 16:35:33.438691 6076 ts_manager.cc:194] Registered new tserver with Master: 3b697ba16edf499e8812522375210c09 (127.19.238.65:39495)
I20260812 16:35:33.440047 6076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:46147
I20260812 16:35:33.440961 6294 heartbeater.cc:503] Master 127.19.238.126:39511 was elected leader, sending a full tablet report...
Aug 12 16:35:33 dist-test-slave-mvvj krb5kdc[5398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552533, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for kudu/127.19.238.65@KRBTEST.COM
I20260812 16:35:33.449059 6302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.439357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:39495 (local address 127.0.0.1:43530)
0812 16:35:33.439598 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:33.439629 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 16:35:33.439706 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:35:33.440014 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:35:33.440019 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:35:33.440385 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:35:33.440453 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:33.440455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.441866 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.441867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:33.442496 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:33.442499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.443140 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.443141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:33.443208 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.443805 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:35:33.443810 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:35:33.445498 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:35:33.447673 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:35:33.447680 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:35:33.447682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:35:33.448012 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:35:33.448261 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:35:33.448262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:35:33.448262 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:35:33.448340 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:35:33.448702 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:35:33.448709 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:35:33.448762 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 16:35:33.448849 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 16:35:33.448960 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":111,"threads_started":1}
I20260812 16:35:33.449112 6303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.439439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43530 (local address 127.19.238.65:39495)
0812 16:35:33.439641 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:33.439646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:33.439688 (+ 42us) server_negotiation.cc:407] Connection header received
0812 16:35:33.439786 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:33.439789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:33.439898 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:33.439987 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:35:33.440547 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.441760 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.442585 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.443055 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.443127 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.445597 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:35:33.445607 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:35:33.445607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:35:33.445615 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:35:33.447580 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:33.448094 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:33.448097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:33.448098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:33.448167 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:33.448412 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:33.448413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:33.448413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:33.448611 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:35:33.448699 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:33.448904 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 16:35:33.449010 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":170,"thread_start_us":101,"threads_started":1}
I20260812 16:35:33.449658 6219 tablet_service.cc:1514] Processing DeleteTablet for tablet b1521454c65c4e3abc3b2f027aa1c214 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:35:13 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:43530
I20260812 16:35:33.450359 6304 tablet_replica.cc:330] stopping tablet replica
I20260812 16:35:33.450526 6304 raft_consensus.cc:2229] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:35:33.450620 6304 raft_consensus.cc:2258] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:35:33.450968 6304 ts_tablet_manager.cc:1905] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:35:33.453209 6304 ts_tablet_manager.cc:1918] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:35:33.453299 6304 log.cc:1196] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/wal/wals/b1521454c65c4e3abc3b2f027aa1c214
I20260812 16:35:33.453547 6304 ts_tablet_manager.cc:1939] T b1521454c65c4e3abc3b2f027aa1c214 P 3b697ba16edf499e8812522375210c09: Deleting consensus metadata
I20260812 16:35:33.454115 6061 catalog_manager.cc:4841] TS 3b697ba16edf499e8812522375210c09 (127.19.238.65:39495): tablet b1521454c65c4e3abc3b2f027aa1c214 (table test-table [id=6170805c4215465085d785d1f9e65f7e]) successfully deleted
W20260812 16:35:33.521703 6301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:35:33.521931 6301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:35:33.521955 6301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:35:33.523133 6301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:35:33.523177 6301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:35:33.523204 6301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:35:33.524708 6301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:37659
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=34795
--tserver_master_addrs=127.19.238.126:39511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:35:33.524966 6301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:35:33.525203 6301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:35:33.530582 6309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:35:33.530728 6301 server_base.cc:1034] running on GCE node
W20260812 16:35:33.530589 6313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:35:33.530601 6310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:35:33.531059 6301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:35:33.531292 6301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:35:33.532477 6301 hybrid_clock.cc:648] HybridClock initialized: now 1786552533532473 us; error 62 us; skew 500 ppm
Aug 12 16:35:33 dist-test-slave-mvvj krb5kdc[5398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552533, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:35:33.535387 6301 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:35:33.535980 6301 webserver.cc:460] Webserver started at http://127.19.238.66:34795/ using document root <none> and password file <none>
I20260812 16:35:33.536227 6301 fs_manager.cc:359] Metadata directory not provided
I20260812 16:35:33.536309 6301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:35:33.537746 6301 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:35:33.538501 6320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:33.538812 6301 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:35:33.538921 6301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "769309b27347411c0b3e6b009bbe143b"
server_key_iv: "5018ef98243e64f478b5bae3c1490930"
server_key_version: "encryptionkey@0"
I20260812 16:35:33.539304 6301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:35:33.548959 6301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:35:33.551270 6301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:35:33.551438 6301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:35:33.551692 6301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:35:33.552084 6301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:35:33.552138 6301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:33.552181 6301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:35:33.552208 6301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:33.558272 6301 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:37659
I20260812 16:35:33.558342 6447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:37659 every 8 connection(s)
I20260812 16:35:33.558709 6301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:35:33.564698 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 6301
I20260812 16:35:33.565068 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:39859
--local_ip_for_outbound_sockets=127.19.238.67
--tserver_master_addrs=127.19.238.126:39511
--webserver_port=45187
--webserver_interface=127.19.238.67
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:35:33 dist-test-slave-mvvj krb5kdc[5398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552533, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:35:33.569116 6298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.559473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:52783 (local address 127.19.238.126:39511)
0812 16:35:33.559522 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:33.559523 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:33.559759 (+ 236us) server_negotiation.cc:407] Connection header received
0812 16:35:33.560305 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:33.560306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:33.560320 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:33.560375 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:35:33.561300 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.561800 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.562799 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.563053 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.563115 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.565886 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:35:33.565888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:35:33.565888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:35:33.565895 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:35:33.567484 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:33.568076 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:33.568077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:33.568078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:33.568135 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:33.568420 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:33.568421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:33.568421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:33.568545 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:35:33.568597 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:33.568949 (+ 352us) server_negotiation.cc:299] Negotiation successful
0812 16:35:33.569047 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:35:33.569393 6450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.559437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:52783)
0812 16:35:33.559624 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:33.559661 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:35:33.560207 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:35:33.560437 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:35:33.560440 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:35:33.561018 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:35:33.561216 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:33.561220 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.561885 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.561886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:33.562715 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:33.562717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.563193 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.563194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:33.563313 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.563918 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:35:33.563926 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:35:33.565766 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:35:33.567573 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:35:33.567581 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:35:33.567583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:35:33.567995 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:35:33.568204 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:35:33.568205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:35:33.568205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:35:33.568354 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:35:33.568607 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:35:33.568614 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:35:33.568795 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 16:35:33.568916 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 16:35:33.569105 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
I20260812 16:35:33.570088 6448 heartbeater.cc:347] Connected to a master server at 127.19.238.126:39511
I20260812 16:35:33.570191 6448 heartbeater.cc:464] Registering TS with master...
I20260812 16:35:33.570394 6448 heartbeater.cc:511] Master 127.19.238.126:39511 requested a full tablet report, sending...
I20260812 16:35:33.570812 6076 ts_manager.cc:194] Registered new tserver with Master: 15cb6a42b7994dc6ae21bd1d0d3062bb (127.19.238.66:37659)
I20260812 16:35:33.571388 6076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:52783
W20260812 16:35:33.650611 6451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:35:33.650837 6451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:35:33.650862 6451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:35:33.652065 6451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:35:33.652110 6451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:35:33.652137 6451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:35:33.653537 6451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:39859
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=45187
--tserver_master_addrs=127.19.238.126:39511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:35:33.653729 6451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:35:33.653911 6451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:35:33.658972 6456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:35:33.659096 6457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:35:33.659224 6451 server_base.cc:1034] running on GCE node
W20260812 16:35:33.659096 6459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:35:33.659551 6451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:35:33.659765 6451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:35:33.660923 6451 hybrid_clock.cc:648] HybridClock initialized: now 1786552533660903 us; error 39 us; skew 500 ppm
Aug 12 16:35:33 dist-test-slave-mvvj krb5kdc[5398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552533, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:35:33.663563 6451 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:35:33.664090 6451 webserver.cc:460] Webserver started at http://127.19.238.67:45187/ using document root <none> and password file <none>
I20260812 16:35:33.664327 6451 fs_manager.cc:359] Metadata directory not provided
I20260812 16:35:33.664414 6451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:35:33.665897 6451 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:35:33.666659 6466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:33.666859 6451 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:35:33.666966 6451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "3f9e8fd85db0448ab43452f87bb4c0e1"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "75c1d9d46c2accbbe4e27eeb3dd4f52f"
server_key_iv: "38222d3ea6ebdcf9f06893d72f243298"
server_key_version: "encryptionkey@0"
I20260812 16:35:33.667382 6451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:35:33.681350 6451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:35:33.684367 6451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:35:33.684542 6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:35:33.684751 6451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:35:33.685081 6451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:35:33.685122 6451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:33.685146 6451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:35:33.685159 6451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:33.691197 6451 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:39859
I20260812 16:35:33.691273 6592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:39859 every 8 connection(s)
I20260812 16:35:33.691663 6451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:35:33.691951 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 6451
Aug 12 16:35:33 dist-test-slave-mvvj krb5kdc[5398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552533, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:35:33.700167 6298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.692423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42826 (local address 127.19.238.126:39511)
0812 16:35:33.692497 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:33.692499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:33.692652 (+ 153us) server_negotiation.cc:407] Connection header received
0812 16:35:33.692755 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:33.692757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:33.692770 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:33.692854 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:35:33.693599 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.694388 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.695300 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.695494 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.695548 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.696981 (+ 1433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:35:33.696983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:35:33.696983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:35:33.696991 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:35:33.698693 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:33.699157 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:33.699158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:33.699159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:33.699226 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:33.699462 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:33.699463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:33.699464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:33.699679 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:35:33.699883 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:33.699963 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 16:35:33.700060 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:35:33.702764 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.692780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:36537 (local address 127.19.238.126:39511)
0812 16:35:33.693178 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:33.693180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:33.693196 (+ 16us) server_negotiation.cc:407] Connection header received
0812 16:35:33.693800 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:33.693801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:33.693812 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:33.693866 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:35:33.694975 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.695697 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.696783 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.697082 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.697135 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.699565 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:35:33.699567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:35:33.699567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:35:33.699574 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:35:33.701353 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:33.701793 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:33.701794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:33.701794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:33.701848 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:33.702145 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:33.702145 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:33.702145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:33.702251 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:35:33.702302 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:33.702610 (+ 308us) server_negotiation.cc:299] Negotiation successful
0812 16:35:33.702691 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":54,"threads_started":1}
I20260812 16:35:33.702960 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.692838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:36537)
0812 16:35:33.693071 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:33.693111 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 16:35:33.693695 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:35:33.693940 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:35:33.693945 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:35:33.694675 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:35:33.694875 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:33.694880 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.695793 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.695794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:33.696692 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:33.696695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.697167 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.697169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:33.697315 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.697708 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:35:33.697718 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:35:33.699478 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:35:33.701477 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:35:33.701485 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:35:33.701487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:35:33.701721 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:35:33.701936 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:35:33.701937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:35:33.701938 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:35:33.702073 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:35:33.702296 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:35:33.702299 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:35:33.702448 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 16:35:33.702550 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 16:35:33.702696 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":112,"threads_started":1}
I20260812 16:35:33.703645 6593 heartbeater.cc:347] Connected to a master server at 127.19.238.126:39511
I20260812 16:35:33.703758 6593 heartbeater.cc:464] Registering TS with master...
I20260812 16:35:33.704013 6593 heartbeater.cc:511] Master 127.19.238.126:39511 requested a full tablet report, sending...
I20260812 16:35:33.704438 6076 ts_manager.cc:194] Registered new tserver with Master: 3f9e8fd85db0448ab43452f87bb4c0e1 (127.19.238.67:39859)
I20260812 16:35:33.704836 6076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:36537
I20260812 16:35:33.712508 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:35:33.715667 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.713060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42834 (local address 127.19.238.126:39511)
0812 16:35:33.713129 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:33.713131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:33.713140 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:35:33.713279 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:33.713280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:33.713325 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:33.713452 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:35:33.713625 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.714144 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.714928 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.715140 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.715200 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.715365 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:35:33.715461 (+ 96us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 16:35:33.715604 (+ 143us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42834: token expired
Metrics: {"server-negotiator.queue_time_us":40}
W20260812 16:35:33.715770 6597 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42834: token expired
W20260812 16:35:33.715934 5993 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.19.238.126:39511)
I20260812 16:35:33.722584 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.716245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42850 (local address 127.19.238.126:39511)
0812 16:35:33.716289 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:33.716291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:33.716304 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:35:33.716322 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:33.716323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:33.716332 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:33.716384 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:35:33.716867 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.717355 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.718128 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.718322 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.718386 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.719770 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:35:33.719771 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:35:33.719771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:35:33.719779 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:35:33.721344 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:33.721763 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:33.721764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:33.721764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:33.721821 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:33.722036 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:33.722038 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:33.722038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:33.722183 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:35:33.722280 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:33.722426 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 16:35:33.722505 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 16:35:33.724058 6076 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:35:33.725778 6076 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:35:33.728238 6219 tablet_service.cc:1467] Processing CreateTablet for tablet 565bc3b8c685444aaef3ab58bb10e15d (DEFAULT_TABLE table=test-table [id=611cd82ffaec4c0081763c682f98c05f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:35:33.728471 6219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 565bc3b8c685444aaef3ab58bb10e15d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:35:33.729957 6170 tablet_bootstrap.cc:492] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09: Bootstrap starting.
I20260812 16:35:33.730315 6170 tablet_bootstrap.cc:654] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09: Neither blocks nor log segments found. Creating new log.
I20260812 16:35:33.730991 6170 tablet_bootstrap.cc:492] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09: No bootstrap required, opened a new log
I20260812 16:35:33.731055 6170 ts_tablet_manager.cc:1397] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 16:35:33.731452 6170 raft_consensus.cc:348] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:33.731537 6170 raft_consensus.cc:374] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:35:33.731601 6170 raft_consensus.cc:729] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b697ba16edf499e8812522375210c09, State: Initialized, Role: FOLLOWER
I20260812 16:35:33.731666 6170 consensus_queue.cc:260] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:33.731776 6170 ts_tablet_manager.cc:1428] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 16:35:33.732285 6600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.727813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:37659 (local address 127.0.0.1:53382)
0812 16:35:33.728056 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:33.728067 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:35:33.728117 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:35:33.728545 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:35:33.728546 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:35:33.728548 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:35:33.728611 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:33.728613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.729682 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.729683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:33.730829 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:33.730833 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.731948 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.731949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:33.732068 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.732085 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 16:35:33.732140 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 16:35:33.732183 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":13,"thread_start_us":88,"threads_started":1}
I20260812 16:35:33.732550 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.728068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53382 (local address 127.19.238.66:37659)
0812 16:35:33.728238 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:33.728243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:33.728254 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:35:33.728299 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:33.728301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:33.728327 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:33.728418 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:35:33.728693 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.729594 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.730999 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.731864 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.731934 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.732000 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:33.732404 (+ 404us) server_negotiation.cc:299] Negotiation successful
0812 16:35:33.732463 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
I20260812 16:35:33.733254 6374 tablet_service.cc:1467] Processing CreateTablet for tablet 565bc3b8c685444aaef3ab58bb10e15d (DEFAULT_TABLE table=test-table [id=611cd82ffaec4c0081763c682f98c05f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:35:33.733680 6374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 565bc3b8c685444aaef3ab58bb10e15d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:35:33.733968 6302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.727762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:39859 (local address 127.0.0.1:55558)
0812 16:35:33.727827 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:33.727859 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 16:35:33.727902 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:35:33.730246 (+ 2344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:35:33.730247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:35:33.730249 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:35:33.730300 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:33.730301 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.731668 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.731669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:33.732890 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:33.732894 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.733682 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.733683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:33.733774 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.733779 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:35:33.733846 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 16:35:33.733894 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 16:35:33.733999 6602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.727873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55558 (local address 127.19.238.67:39859)
0812 16:35:33.730066 (+ 2193us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:33.730071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:33.730083 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:35:33.730131 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:33.730134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:33.730151 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:33.730236 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:35:33.730360 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.731543 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.733015 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.733591 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.733683 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.733747 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:33.733862 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 16:35:33.733906 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2144,"thread_start_us":139,"threads_started":1}
I20260812 16:35:33.736775 6605 tablet_bootstrap.cc:492] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb: Bootstrap starting.
I20260812 16:35:33.736716 6521 tablet_service.cc:1467] Processing CreateTablet for tablet 565bc3b8c685444aaef3ab58bb10e15d (DEFAULT_TABLE table=test-table [id=611cd82ffaec4c0081763c682f98c05f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:35:33.737097 6521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 565bc3b8c685444aaef3ab58bb10e15d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:35:33.737636 6605 tablet_bootstrap.cc:654] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb: Neither blocks nor log segments found. Creating new log.
I20260812 16:35:33.738111 6605 log.cc:824] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb: Log is configured to *not* fsync() on all Append() calls
I20260812 16:35:33.738875 6605 tablet_bootstrap.cc:492] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb: No bootstrap required, opened a new log
I20260812 16:35:33.738940 6605 ts_tablet_manager.cc:1397] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:35:33.740054 6607 tablet_bootstrap.cc:492] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1: Bootstrap starting.
I20260812 16:35:33.740835 6607 tablet_bootstrap.cc:654] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1: Neither blocks nor log segments found. Creating new log.
I20260812 16:35:33.740890 6605 raft_consensus.cc:348] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:33.741114 6605 raft_consensus.cc:374] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:35:33.741161 6605 raft_consensus.cc:729] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15cb6a42b7994dc6ae21bd1d0d3062bb, State: Initialized, Role: FOLLOWER
I20260812 16:35:33.741314 6605 consensus_queue.cc:260] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:33.741508 6607 log.cc:824] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1: Log is configured to *not* fsync() on all Append() calls
I20260812 16:35:33.741660 6605 ts_tablet_manager.cc:1428] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 16:35:33.741683 6448 heartbeater.cc:503] Master 127.19.238.126:39511 was elected leader, sending a full tablet report...
I20260812 16:35:33.742488 6607 tablet_bootstrap.cc:492] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1: No bootstrap required, opened a new log
I20260812 16:35:33.742573 6607 ts_tablet_manager.cc:1397] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:35:33.744446 6607 raft_consensus.cc:348] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:33.744603 6607 raft_consensus.cc:374] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:35:33.744627 6607 raft_consensus.cc:729] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f9e8fd85db0448ab43452f87bb4c0e1, State: Initialized, Role: FOLLOWER
I20260812 16:35:33.744761 6607 consensus_queue.cc:260] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:33.745114 6593 heartbeater.cc:503] Master 127.19.238.126:39511 was elected leader, sending a full tablet report...
I20260812 16:35:33.745116 6607 ts_tablet_manager.cc:1428] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260812 16:35:33.809333 6449 tablet.cc:2369] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:35:33.862145 6300 raft_consensus.cc:482] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:35:33.862308 6300 raft_consensus.cc:504] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:33.862728 6300 leader_election.cc:288] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f9e8fd85db0448ab43452f87bb4c0e1 (127.19.238.67:39859), 15cb6a42b7994dc6ae21bd1d0d3062bb (127.19.238.66:37659)
I20260812 16:35:33.867323 6297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.862934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:39859 (local address 127.19.238.65:34945)
0812 16:35:33.862979 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:33.862994 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:35:33.863064 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:35:33.863389 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:35:33.863391 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:35:33.863393 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:35:33.863493 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:33.863494 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.864545 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.864546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:33.866121 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:33.866122 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.866959 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.866960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:33.867112 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.867115 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:35:33.867173 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 16:35:33.867212 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":124}
I20260812 16:35:33.867349 6602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.863097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:34945 (local address 127.19.238.67:39859)
0812 16:35:33.863164 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:33.863166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:33.863180 (+ 14us) server_negotiation.cc:407] Connection header received
0812 16:35:33.863207 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:33.863208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:33.863215 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:33.863284 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:35:33.863571 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.864341 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.866239 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.866835 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.866926 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.866936 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:33.867235 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 16:35:33.867281 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 16:35:33.867349 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.863075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:42781 (local address 127.19.238.66:37659)
0812 16:35:33.863124 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:33.863127 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:33.863264 (+ 137us) server_negotiation.cc:407] Connection header received
0812 16:35:33.863328 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:33.863329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:33.863341 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:33.863389 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:35:33.863570 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.864341 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.866239 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.866854 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.866926 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.866936 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:33.867225 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 16:35:33.867277 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 16:35:33.867579 6611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.862992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:37659 (local address 127.19.238.65:42781)
0812 16:35:33.863186 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:33.863195 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:35:33.863257 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:35:33.863408 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:35:33.863409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:35:33.863410 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:35:33.863492 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:33.863494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.864544 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.864546 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:33.866109 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:33.866113 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.866970 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.866971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:33.867050 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.867090 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 16:35:33.867162 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 16:35:33.867207 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":138,"thread_start_us":114,"threads_started":1}
I20260812 16:35:33.867942 6544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "565bc3b8c685444aaef3ab58bb10e15d" candidate_uuid: "3b697ba16edf499e8812522375210c09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" is_pre_election: true
I20260812 16:35:33.868011 6397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "565bc3b8c685444aaef3ab58bb10e15d" candidate_uuid: "3b697ba16edf499e8812522375210c09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" is_pre_election: true
I20260812 16:35:33.868129 6544 raft_consensus.cc:2454] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b697ba16edf499e8812522375210c09 in term 0.
I20260812 16:35:33.868178 6397 raft_consensus.cc:2454] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b697ba16edf499e8812522375210c09 in term 0.
I20260812 16:35:33.868358 6167 leader_election.cc:302] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b697ba16edf499e8812522375210c09, 3f9e8fd85db0448ab43452f87bb4c0e1; no voters:
I20260812 16:35:33.868526 6300 raft_consensus.cc:2781] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:35:33.868588 6300 raft_consensus.cc:482] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:35:33.868613 6300 raft_consensus.cc:3037] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:35:33.869360 6300 raft_consensus.cc:504] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:33.869517 6300 leader_election.cc:288] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 1 election: Requested vote from peers 3f9e8fd85db0448ab43452f87bb4c0e1 (127.19.238.67:39859), 15cb6a42b7994dc6ae21bd1d0d3062bb (127.19.238.66:37659)
I20260812 16:35:33.869678 6544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "565bc3b8c685444aaef3ab58bb10e15d" candidate_uuid: "3b697ba16edf499e8812522375210c09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1"
I20260812 16:35:33.869715 6397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "565bc3b8c685444aaef3ab58bb10e15d" candidate_uuid: "3b697ba16edf499e8812522375210c09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb"
I20260812 16:35:33.869781 6544 raft_consensus.cc:3037] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:35:33.869786 6397 raft_consensus.cc:3037] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:35:33.870731 6544 raft_consensus.cc:2454] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b697ba16edf499e8812522375210c09 in term 1.
I20260812 16:35:33.870731 6397 raft_consensus.cc:2454] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b697ba16edf499e8812522375210c09 in term 1.
I20260812 16:35:33.870909 6167 leader_election.cc:302] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b697ba16edf499e8812522375210c09, 3f9e8fd85db0448ab43452f87bb4c0e1; no voters:
I20260812 16:35:33.871064 6300 raft_consensus.cc:2781] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:35:33.871256 6300 raft_consensus.cc:686] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 LEADER]: Becoming Leader. State: Replica: 3b697ba16edf499e8812522375210c09, State: Running, Role: LEADER
I20260812 16:35:33.871371 6300 consensus_queue.cc:237] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:33.872310 6076 catalog_manager.cc:5495] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b697ba16edf499e8812522375210c09 (127.19.238.65). New cstate: current_term: 1 leader_uuid: "3b697ba16edf499e8812522375210c09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } health_report { overall_health: HEALTHY } } }
I20260812 16:35:33.878019 6303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.875697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43540 (local address 127.19.238.65:39495)
0812 16:35:33.875757 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:33.875759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:33.875767 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:35:33.875818 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:33.875818 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:33.875882 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:33.875963 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:35:33.876059 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.876574 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.877291 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:33.877536 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:33.877593 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:33.877719 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:35:33.877815 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:35:33.877871 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:33.877902 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 16:35:33.877922 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 16:35:33.879565 6397 raft_consensus.cc:1261] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb [term 1 FOLLOWER]: Refusing update from remote peer 3b697ba16edf499e8812522375210c09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:35:33.879590 6544 raft_consensus.cc:1261] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1 [term 1 FOLLOWER]: Refusing update from remote peer 3b697ba16edf499e8812522375210c09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:35:33.879973 6612 consensus_queue.cc:1035] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:35:33.880100 6300 consensus_queue.cc:1035] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:35:33.884531 6616 mvcc.cc:205] Tried to move back new op lower bound from 7317719178767917056 to 7317719178738016256. Current Snapshot: MvccSnapshot[applied={T|T < 7317719178767917056}]
I20260812 16:35:33.886833 6076 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:35:33.886951 6076 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:35:33.887694 6076 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: Sending DeleteTablet for 3 replicas of tablet 565bc3b8c685444aaef3ab58bb10e15d
I20260812 16:35:33.888072 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 6118
I20260812 16:35:33.888073 6374 tablet_service.cc:1514] Processing DeleteTablet for tablet 565bc3b8c685444aaef3ab58bb10e15d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:35:33 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:53382
I20260812 16:35:33.888118 6521 tablet_service.cc:1514] Processing DeleteTablet for tablet 565bc3b8c685444aaef3ab58bb10e15d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:35:33 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:55558
I20260812 16:35:33.890051 6625 tablet_replica.cc:330] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb: stopping tablet replica
I20260812 16:35:33.890141 6626 tablet_replica.cc:330] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1: stopping tablet replica
I20260812 16:35:33.890259 6625 raft_consensus.cc:2229] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:35:33.890259 6626 raft_consensus.cc:2229] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:35:33.890380 6626 raft_consensus.cc:2258] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:35:33.890386 6625 raft_consensus.cc:2258] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:35:33.890887 6625 ts_tablet_manager.cc:1905] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:35:33.890888 6626 ts_tablet_manager.cc:1905] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:35:33.893039 6626 ts_tablet_manager.cc:1918] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:35:33.893039 6625 ts_tablet_manager.cc:1918] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:35:33.893128 6625 log.cc:1196] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/565bc3b8c685444aaef3ab58bb10e15d
I20260812 16:35:33.893128 6626 log.cc:1196] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/565bc3b8c685444aaef3ab58bb10e15d
I20260812 16:35:33.893466 6626 ts_tablet_manager.cc:1939] T 565bc3b8c685444aaef3ab58bb10e15d P 3f9e8fd85db0448ab43452f87bb4c0e1: Deleting consensus metadata
I20260812 16:35:33.893507 6625 ts_tablet_manager.cc:1939] T 565bc3b8c685444aaef3ab58bb10e15d P 15cb6a42b7994dc6ae21bd1d0d3062bb: Deleting consensus metadata
I20260812 16:35:33.893942 6062 catalog_manager.cc:4841] TS 3f9e8fd85db0448ab43452f87bb4c0e1 (127.19.238.67:39859): tablet 565bc3b8c685444aaef3ab58bb10e15d (table test-table [id=611cd82ffaec4c0081763c682f98c05f]) successfully deleted
I20260812 16:35:33.893973 6059 catalog_manager.cc:4841] TS 15cb6a42b7994dc6ae21bd1d0d3062bb (127.19.238.66:37659): tablet 565bc3b8c685444aaef3ab58bb10e15d (table test-table [id=611cd82ffaec4c0081763c682f98c05f]) successfully deleted
W20260812 16:35:33.897114 6061 connection.cc:535] client connection to 127.19.238.65:39495 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 16:35:33.897284 6061 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 16:35:33.897724 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 6301
I20260812 16:35:33.897921 6302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:33.897734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:39495 (local address 127.0.0.1:43546)
0812 16:35:33.897818 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:33.897879 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:39495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 16:35:33.898923 6061 catalog_manager.cc:4584] TS 3b697ba16edf499e8812522375210c09 (127.19.238.65:39495): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 565bc3b8c685444aaef3ab58bb10e15d: Network error: Client connection negotiation failed: client connection to 127.19.238.65:39495: connect: Connection refused (error 111)
I20260812 16:35:33.906636 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 6451
I20260812 16:35:33.914058 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 6042
I20260812 16:35:53.921417 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:39511
--webserver_interface=127.19.238.126
--webserver_port=33421
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:39511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:35:54.009281 6628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:35:54.009531 6628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:35:54.009557 6628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:35:54.010831 6628 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 16:35:54.010883 6628 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 16:35:54.010896 6628 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 16:35:54.010908 6628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:35:54.010922 6628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:35:54.010936 6628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:35:54.010977 6628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:35:54.011010 6628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:35:54.012717 6628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:39511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:39511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=33421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:35:54.013048 6628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:35:54.013320 6628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:35:54.019184 6635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:35:54.019167 6638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:35:54.019167 6634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:35:54.019245 6628 server_base.cc:1034] running on GCE node
I20260812 16:35:54.019748 6628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:35:54.020118 6628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:35:54.021323 6628 hybrid_clock.cc:648] HybridClock initialized: now 1786552554021305 us; error 49 us; skew 500 ppm
Aug 12 16:35:54 dist-test-slave-mvvj krb5kdc[5398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552554, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:35:54.028626 6628 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:35:54.029150 6628 webserver.cc:460] Webserver started at http://127.19.238.126:33421/ using document root <none> and password file <none>
I20260812 16:35:54.029414 6628 fs_manager.cc:359] Metadata directory not provided
I20260812 16:35:54.029490 6628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:35:54.031013 6628 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:35:54.031780 6645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:54.031999 6628 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:35:54.032121 6628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "789ea6d5a384a32b17b6dabdffaa5241"
server_key_iv: "767d5771c849a281f9c01dd4dbff285a"
server_key_version: "encryptionkey@0"
I20260812 16:35:54.032596 6628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:35:54.053447 6628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:35:54.055763 6628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:35:54.055969 6628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:35:54.060840 6628 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:39511
I20260812 16:35:54.060880 6702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:39511 every 8 connection(s)
I20260812 16:35:54.061304 6628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:35:54.062855 6703 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:35:54.063594 6703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: Bootstrap starting.
I20260812 16:35:54.068176 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 6628
I20260812 16:35:54.068506 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:39495
--local_ip_for_outbound_sockets=127.19.238.65
--tserver_master_addrs=127.19.238.126:39511
--webserver_port=34403
--webserver_interface=127.19.238.65
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:35:54.073467 6703 log.cc:824] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:35:54.076006 6703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:35:54.076371 6703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: Bootstrap complete.
I20260812 16:35:54.078805 6703 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39511 } }
I20260812 16:35:54.079054 6703 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f0f7f77b7134f0ba41c13b0f75c2e4e, State: Initialized, Role: FOLLOWER
I20260812 16:35:54.079236 6703 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39511 } }
I20260812 16:35:54.079313 6703 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:35:54.079376 6703 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:35:54.079422 6703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 2 FOLLOWER]: Advancing to term 3
I20260812 16:35:54.080653 6703 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39511 } }
I20260812 16:35:54.080749 6703 leader_election.cc:302] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f0f7f77b7134f0ba41c13b0f75c2e4e; no voters:
I20260812 16:35:54.080972 6703 leader_election.cc:288] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 16:35:54.081014 6707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 3 FOLLOWER]: Leader election won for term 3
I20260812 16:35:54.081214 6707 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [term 3 LEADER]: Becoming Leader. State: Replica: 4f0f7f77b7134f0ba41c13b0f75c2e4e, State: Running, Role: LEADER
I20260812 16:35:54.081312 6703 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:35:54.081377 6707 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39511 } }
I20260812 16:35:54.081743 6708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39511 } } }
I20260812 16:35:54.081838 6708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [sys.catalog]: This master's current role is: LEADER
I20260812 16:35:54.082080 6709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f0f7f77b7134f0ba41c13b0f75c2e4e. Latest consensus state: current_term: 3 leader_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f0f7f77b7134f0ba41c13b0f75c2e4e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 39511 } } }
I20260812 16:35:54.082193 6709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e [sys.catalog]: This master's current role is: LEADER
I20260812 16:35:54.082327 6714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:35:54.083182 6714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:35:54.083284 6714 catalog_manager.cc:1259] Loaded cluster ID: 63c6237c128b4531ade0c866dc5058f2
I20260812 16:35:54.083371 6714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:35:54.084602 6714 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:35:54.084789 6714 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: Loaded TSK: 0
I20260812 16:35:54.084841 6714 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4f0f7f77b7134f0ba41c13b0f75c2e4e: Loaded TSK: 1
F20260812 16:35:54.085088 6714 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2fd674b9ad google::LogMessage::Fail() at ??:0
@ 0x7f2fd674e88a google::LogMessage::SendToLog() at ??:0
@ 0x7f2fd674b4de google::LogMessage::Flush() at ??:0
@ 0x7f2fd674ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2fd8eb8104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f2fd8eb5d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f2fde6a51cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f2fde6a548e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f2fde6a5594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f2fde66bc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f2fde675ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2fd86190d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2fd860e954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2fd94726db start_thread at ??:0
@ 0x7f2fd524961f clone at ??:0
W20260812 16:35:54.174454 6705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:35:54.174750 6705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:35:54.174795 6705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:35:54.176247 6705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:35:54.176299 6705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:35:54.176330 6705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:35:54.178081 6705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:39495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=34403
--tserver_master_addrs=127.19.238.126:39511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:35:54.178398 6705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:35:54.178664 6705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:35:54.184156 6749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:35:54.184150 6752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:35:54.184150 6750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:35:54.184700 6705 server_base.cc:1034] running on GCE node
I20260812 16:35:54.184998 6705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:35:54.185321 6705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:35:54.186522 6705 hybrid_clock.cc:648] HybridClock initialized: now 1786552554186502 us; error 38 us; skew 500 ppm
Aug 12 16:35:54 dist-test-slave-mvvj krb5kdc[5398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552554, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:35:54.189790 6705 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:35:54.190305 6705 webserver.cc:460] Webserver started at http://127.19.238.65:34403/ using document root <none> and password file <none>
I20260812 16:35:54.190572 6705 fs_manager.cc:359] Metadata directory not provided
I20260812 16:35:54.190651 6705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:35:54.192214 6705 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:35:54.193006 6761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:54.193213 6705 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:35:54.193325 6705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "3b697ba16edf499e8812522375210c09"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "990352e8cb25fa958bfdfa823398fc91"
server_key_iv: "469226635dfbe525da629d23e6cc2532"
server_key_version: "encryptionkey@0"
I20260812 16:35:54.193744 6705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:35:54.205480 6705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:35:54.208011 6705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:35:54.208204 6705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:35:54.208503 6705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:35:54.209004 6769 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 16:35:54.211236 6705 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 16:35:54.211319 6705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 16:35:54.211369 6705 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 16:35:54.212159 6705 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 16:35:54.212217 6705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260812 16:35:54.212301 6769 tablet_bootstrap.cc:492] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09: Bootstrap starting.
I20260812 16:35:54.219959 6705 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:39495
I20260812 16:35:54.220013 6886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:39495 every 8 connection(s)
I20260812 16:35:54.220474 6705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:35:54.224411 6769 log.cc:824] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09: Log is configured to *not* fsync() on all Append() calls
I20260812 16:35:54.225652 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 6705
I20260812 16:35:54.225960 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:37659
--local_ip_for_outbound_sockets=127.19.238.66
--tserver_master_addrs=127.19.238.126:39511
--webserver_port=34795
--webserver_interface=127.19.238.66
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:35:54.227335 6769 tablet_bootstrap.cc:492] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:35:54.227761 6769 tablet_bootstrap.cc:492] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09: Bootstrap complete.
I20260812 16:35:54.229135 6769 ts_tablet_manager.cc:1397] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.010s
I20260812 16:35:54.230842 6769 raft_consensus.cc:348] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:54.231205 6769 raft_consensus.cc:729] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b697ba16edf499e8812522375210c09, State: Initialized, Role: FOLLOWER
I20260812 16:35:54.231397 6769 consensus_queue.cc:260] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:54.231709 6769 ts_tablet_manager.cc:1428] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
W20260812 16:35:54.308107 6889 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:35:54.221273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:39467)
0812 16:35:54.221461 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:54.221508 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 16:35:54.221993 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:35:54.307862 (+ 85869us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
W20260812 16:35:54.308434 6887 heartbeater.cc:644] Failed to heartbeat to 127.19.238.126:39511 (0 consecutive failures): Network error: Failed to ping master at 127.19.238.126:39511: Client connection negotiation failed: client connection to 127.19.238.126:39511: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 16:35:54.308876 6889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:54.308732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:33389)
0812 16:35:54.308797 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:54.308850 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 16:35:54.309273 6889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:54.309168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:42061)
0812 16:35:54.309218 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:54.309249 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 16:35:54.309433 6887 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 16:35:54.314324 6892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:35:54.314545 6892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:35:54.314604 6892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:35:54.315933 6892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:35:54.315984 6892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:35:54.316033 6892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:35:54.317646 6892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:37659
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=34795
--tserver_master_addrs=127.19.238.126:39511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:35:54.317919 6892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:35:54.318181 6892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:35:54.323625 6902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:35:54.323625 6904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:35:54.323647 6901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:35:54.323678 6892 server_base.cc:1034] running on GCE node
I20260812 16:35:54.324100 6892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:35:54.324391 6892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:35:54.325557 6892 hybrid_clock.cc:648] HybridClock initialized: now 1786552554325546 us; error 29 us; skew 500 ppm
Aug 12 16:35:54 dist-test-slave-mvvj krb5kdc[5398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552554, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:35:54.328253 6892 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:35:54.328778 6892 webserver.cc:460] Webserver started at http://127.19.238.66:34795/ using document root <none> and password file <none>
I20260812 16:35:54.329046 6892 fs_manager.cc:359] Metadata directory not provided
I20260812 16:35:54.329120 6892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:35:54.330550 6892 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:35:54.331200 6912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:54.331398 6892 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:35:54.331504 6892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "769309b27347411c0b3e6b009bbe143b"
server_key_iv: "5018ef98243e64f478b5bae3c1490930"
server_key_version: "encryptionkey@0"
I20260812 16:35:54.331926 6892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:35:54.341572 6892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:35:54.343688 6892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:35:54.343917 6892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:35:54.344188 6892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:35:54.344544 6892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:35:54.344601 6892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:54.344648 6892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:35:54.344676 6892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:54.351192 6892 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:37659
I20260812 16:35:54.351243 7039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:37659 every 8 connection(s)
I20260812 16:35:54.351616 6892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:35:54.352807 7043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:54.352384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:55031)
0812 16:35:54.352571 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:54.352651 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
W20260812 16:35:54.353332 7040 heartbeater.cc:644] Failed to heartbeat to 127.19.238.126:39511 (0 consecutive failures): Network error: Failed to ping master at 127.19.238.126:39511: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
I20260812 16:35:54.353801 7043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:54.353682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:47241)
0812 16:35:54.353739 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:54.353774 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 16:35:54.354287 7043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:54.354156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:35375)
0812 16:35:54.354227 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:54.354263 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 16:35:54.354468 7040 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 16:35:54.354954 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 6892
I20260812 16:35:54.355230 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:39859
--local_ip_for_outbound_sockets=127.19.238.67
--tserver_master_addrs=127.19.238.126:39511
--webserver_port=45187
--webserver_interface=127.19.238.67
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:35:54.443480 7044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:35:54.443728 7044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:35:54.443756 7044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:35:54.445036 7044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:35:54.445091 7044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:35:54.445119 7044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:35:54.446586 7044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:39859
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=45187
--tserver_master_addrs=127.19.238.126:39511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:35:54.446796 7044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:35:54.446998 7044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:35:54.452397 7050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:35:54.452391 7052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:35:54.452389 7049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:35:54.452607 7044 server_base.cc:1034] running on GCE node
I20260812 16:35:54.452872 7044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:35:54.453159 7044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:35:54.454344 7044 hybrid_clock.cc:648] HybridClock initialized: now 1786552554454324 us; error 32 us; skew 500 ppm
Aug 12 16:35:54 dist-test-slave-mvvj krb5kdc[5398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552554, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:35:54.457350 7044 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:35:54.457948 7044 webserver.cc:460] Webserver started at http://127.19.238.67:45187/ using document root <none> and password file <none>
I20260812 16:35:54.458199 7044 fs_manager.cc:359] Metadata directory not provided
I20260812 16:35:54.458271 7044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:35:54.459882 7044 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:35:54.460593 7059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:54.460769 7044 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:35:54.460877 7044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "3f9e8fd85db0448ab43452f87bb4c0e1"
format_stamp: "Formatted at 2026-08-12 16:35:12 on dist-test-slave-mvvj"
server_key: "75c1d9d46c2accbbe4e27eeb3dd4f52f"
server_key_iv: "38222d3ea6ebdcf9f06893d72f243298"
server_key_version: "encryptionkey@0"
I20260812 16:35:54.461445 7044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:35:54.471778 7044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:35:54.473949 7044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:35:54.474136 7044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:35:54.474398 7044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:35:54.474737 7044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:35:54.474794 7044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:54.474843 7044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:35:54.474866 7044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:35:54.481236 7044 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:39859
I20260812 16:35:54.481287 7183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:39859 every 8 connection(s)
I20260812 16:35:54.481664 7044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:35:54.482828 7187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:54.482421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:33577)
0812 16:35:54.482600 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:54.482677 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
W20260812 16:35:54.483280 7184 heartbeater.cc:644] Failed to heartbeat to 127.19.238.126:39511 (0 consecutive failures): Network error: Failed to ping master at 127.19.238.126:39511: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
I20260812 16:35:54.483604 7187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:54.483477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:54601)
0812 16:35:54.483557 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:54.483585 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 16:35:54.483933 7187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:54.483818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:35403)
0812 16:35:54.483883 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:54.483909 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 16:35:54.484105 7184 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 16:35:54.492005 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 7044
I20260812 16:35:55.310119 7189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:55.309867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:45723)
0812 16:35:55.310010 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:55.310054 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":69,"threads_started":1}
I20260812 16:35:55.355188 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:55.354913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:37407)
0812 16:35:55.355075 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:55.355121 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":66,"threads_started":1}
I20260812 16:35:55.484730 7191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:55.484507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:35811)
0812 16:35:55.484632 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:55.484672 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
I20260812 16:35:56.082152 7192 raft_consensus.cc:482] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:35:56.082352 7192 raft_consensus.cc:504] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:56.082795 7192 leader_election.cc:288] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f9e8fd85db0448ab43452f87bb4c0e1 (127.19.238.67:39859), 15cb6a42b7994dc6ae21bd1d0d3062bb (127.19.238.66:37659)
Aug 12 16:35:56 dist-test-slave-mvvj krb5kdc[5398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552554, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.66@KRBTEST.COM
Aug 12 16:35:56 dist-test-slave-mvvj krb5kdc[5398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552554, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.67@KRBTEST.COM
I20260812 16:35:56.093721 7197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:56.083169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:57709 (local address 127.19.238.67:39859)
0812 16:35:56.083325 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:56.083337 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:56.083362 (+ 25us) server_negotiation.cc:407] Connection header received
0812 16:35:56.083903 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:56.083906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:56.083960 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:56.084110 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:35:56.084893 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:56.085771 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:56.086785 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:56.087206 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:56.087336 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:56.090311 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:35:56.090315 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:35:56.090316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:35:56.090327 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:35:56.092182 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:56.092555 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:56.092558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:56.092558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:56.092674 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:56.092884 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:56.092885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:56.092885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:56.093186 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:35:56.093258 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:56.093455 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 16:35:56.093601 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260812 16:35:56.093734 7195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:56.083130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:40737 (local address 127.19.238.66:37659)
0812 16:35:56.083350 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 16:35:56.083359 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:35:56.083377 (+ 18us) server_negotiation.cc:407] Connection header received
0812 16:35:56.083948 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:35:56.083951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:35:56.083992 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:35:56.084132 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:35:56.084892 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:56.085805 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:56.086798 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:35:56.087208 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:35:56.087345 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:56.090052 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:35:56.090057 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:35:56.090057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:35:56.090070 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:35:56.091779 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:56.092282 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:56.092286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:56.092286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:56.092392 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:35:56.092674 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:35:56.092675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:35:56.092675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:35:56.093018 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:35:56.093119 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 16:35:56.093463 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 16:35:56.093601 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":62,"threads_started":1}
I20260812 16:35:56.093730 7194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:56.083031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:37659 (local address 127.19.238.65:40737)
0812 16:35:56.083187 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:56.083196 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:35:56.083245 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:35:56.084158 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:35:56.084161 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:35:56.084576 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:35:56.084767 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:56.084773 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:56.085909 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:56.085910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:56.086688 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:56.086691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:56.087312 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:56.087313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:56.087438 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:56.088166 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:35:56.088175 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:35:56.089813 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:35:56.091911 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:35:56.091919 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:35:56.091921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:35:56.092173 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:35:56.092465 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:35:56.092466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:35:56.092466 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:35:56.092604 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:35:56.093111 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:35:56.093117 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:35:56.093260 (+ 143us) client_negotiation.cc:769] Sending connection context
0812 16:35:56.093383 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 16:35:56.093605 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":70,"thread_start_us":91,"threads_started":1}
I20260812 16:35:56.093786 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:56.082990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:39859 (local address 127.19.238.65:57709)
0812 16:35:56.083130 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:56.083142 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:35:56.083219 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:35:56.084157 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:35:56.084161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:35:56.084561 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:35:56.084767 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:56.084773 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:56.085886 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:56.085887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:56.086689 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:35:56.086691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:35:56.087308 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:35:56.087309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:35:56.087438 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:35:56.088171 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:35:56.088176 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:35:56.090197 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:35:56.092292 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:35:56.092293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:35:56.092294 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:35:56.092471 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:35:56.092739 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:35:56.092739 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:35:56.092739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:35:56.092793 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:35:56.093267 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:35:56.093268 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:35:56.093303 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 16:35:56.093383 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 16:35:56.093663 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":139,"thread_start_us":95,"threads_started":1}
W20260812 16:35:56.094905 6762 leader_election.cc:341] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 15cb6a42b7994dc6ae21bd1d0d3062bb (127.19.238.66:37659): Not found: Tablet not found: 565bc3b8c685444aaef3ab58bb10e15d
W20260812 16:35:56.095415 6765 leader_election.cc:341] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f9e8fd85db0448ab43452f87bb4c0e1 (127.19.238.67:39859): Not found: Tablet not found: 565bc3b8c685444aaef3ab58bb10e15d
I20260812 16:35:56.095504 6765 leader_election.cc:302] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b697ba16edf499e8812522375210c09; no voters: 15cb6a42b7994dc6ae21bd1d0d3062bb, 3f9e8fd85db0448ab43452f87bb4c0e1
I20260812 16:35:56.095624 7192 raft_consensus.cc:2726] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:35:56.310879 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:56.310723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:46419)
0812 16:35:56.310791 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:56.310835 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 16:35:56.356172 7198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:56.355893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:35757)
0812 16:35:56.356062 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:56.356119 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":87,"threads_started":1}
I20260812 16:35:56.485541 7199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:56.485272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:56071)
0812 16:35:56.485400 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:56.485464 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
I20260812 16:35:57.311767 7200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:57.311481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:58683)
0812 16:35:57.311672 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:57.311717 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":99,"threads_started":1}
I20260812 16:35:57.357036 7201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:57.356731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:39265)
0812 16:35:57.356902 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:57.356969 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":109,"threads_started":1}
I20260812 16:35:57.486290 7202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:57.486075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:33283)
0812 16:35:57.486205 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:57.486242 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260812 16:35:57.788362 7203 raft_consensus.cc:482] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:35:57.788451 7203 raft_consensus.cc:504] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:57.788622 7203 leader_election.cc:288] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f9e8fd85db0448ab43452f87bb4c0e1 (127.19.238.67:39859), 15cb6a42b7994dc6ae21bd1d0d3062bb (127.19.238.66:37659)
W20260812 16:35:57.789149 6765 leader_election.cc:341] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f9e8fd85db0448ab43452f87bb4c0e1 (127.19.238.67:39859): Not found: Tablet not found: 565bc3b8c685444aaef3ab58bb10e15d
W20260812 16:35:57.789238 6762 leader_election.cc:341] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 15cb6a42b7994dc6ae21bd1d0d3062bb (127.19.238.66:37659): Not found: Tablet not found: 565bc3b8c685444aaef3ab58bb10e15d
I20260812 16:35:57.789268 6765 leader_election.cc:302] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b697ba16edf499e8812522375210c09; no voters: 15cb6a42b7994dc6ae21bd1d0d3062bb, 3f9e8fd85db0448ab43452f87bb4c0e1
I20260812 16:35:57.789333 7203 raft_consensus.cc:2726] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:35:58.312563 7204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:58.312318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:42881)
0812 16:35:58.312473 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:58.312513 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":79,"threads_started":1}
I20260812 16:35:58.357841 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:58.357557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:37867)
0812 16:35:58.357738 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:58.357790 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":66,"threads_started":1}
I20260812 16:35:58.487046 7206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:58.486802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:39679)
0812 16:35:58.486945 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:58.486988 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260812 16:35:59.313503 7207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:59.313273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:45961)
0812 16:35:59.313421 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:59.313456 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
I20260812 16:35:59.358657 7208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:59.358409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:47973)
0812 16:35:59.358563 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:59.358609 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":72,"threads_started":1}
I20260812 16:35:59.487686 7209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:35:59.487477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:50709)
0812 16:35:59.487612 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 16:35:59.487643 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260812 16:35:59.880076 7210 raft_consensus.cc:482] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:35:59.880215 7210 raft_consensus.cc:504] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:35:59.880421 7210 leader_election.cc:288] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f9e8fd85db0448ab43452f87bb4c0e1 (127.19.238.67:39859), 15cb6a42b7994dc6ae21bd1d0d3062bb (127.19.238.66:37659)
W20260812 16:35:59.880847 6765 leader_election.cc:341] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f9e8fd85db0448ab43452f87bb4c0e1 (127.19.238.67:39859): Not found: Tablet not found: 565bc3b8c685444aaef3ab58bb10e15d
W20260812 16:35:59.880940 6762 leader_election.cc:341] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 15cb6a42b7994dc6ae21bd1d0d3062bb (127.19.238.66:37659): Not found: Tablet not found: 565bc3b8c685444aaef3ab58bb10e15d
I20260812 16:35:59.881008 6762 leader_election.cc:302] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b697ba16edf499e8812522375210c09; no voters: 15cb6a42b7994dc6ae21bd1d0d3062bb, 3f9e8fd85db0448ab43452f87bb4c0e1
I20260812 16:35:59.881098 7210 raft_consensus.cc:2726] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:36:00.314294 7211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:00.314033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:42245)
0812 16:36:00.314191 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:00.314235 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":69,"threads_started":1}
I20260812 16:36:00.359431 7212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:00.359218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:41827)
0812 16:36:00.359342 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:00.359374 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260812 16:36:00.488454 7213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:00.488219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:45623)
0812 16:36:00.488357 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:00.488404 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
I20260812 16:36:01.315174 7214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:01.314909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:56353)
0812 16:36:01.315074 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:01.315123 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":79,"threads_started":1}
I20260812 16:36:01.360270 7215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:01.360035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:42443)
0812 16:36:01.360181 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:01.360220 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":70,"threads_started":1}
I20260812 16:36:01.489298 7216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:01.489010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:55249)
0812 16:36:01.489196 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:01.489246 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":67,"threads_started":1}
I20260812 16:36:02.316026 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:02.315761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:60813)
0812 16:36:02.315936 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:02.315979 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":115,"threads_started":1}
I20260812 16:36:02.361073 7218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:02.360806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:45299)
0812 16:36:02.360949 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:02.361022 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":67,"threads_started":1}
I20260812 16:36:02.490123 7219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:02.489872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:33133)
0812 16:36:02.490037 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:02.490076 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
I20260812 16:36:03.316774 7220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:03.316549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:34415)
0812 16:36:03.316683 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:03.316725 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":69,"threads_started":1}
I20260812 16:36:03.361867 7221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:03.361637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:35347)
0812 16:36:03.361773 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:03.361820 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":67,"threads_started":1}
I20260812 16:36:03.490967 7222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:03.490647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:38355)
0812 16:36:03.490826 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:03.490889 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":93,"threads_started":1}
I20260812 16:36:03.932313 7223 raft_consensus.cc:482] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:36:03.932406 7223 raft_consensus.cc:504] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:36:03.932586 7223 leader_election.cc:288] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f9e8fd85db0448ab43452f87bb4c0e1 (127.19.238.67:39859), 15cb6a42b7994dc6ae21bd1d0d3062bb (127.19.238.66:37659)
W20260812 16:36:03.932945 6765 leader_election.cc:341] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f9e8fd85db0448ab43452f87bb4c0e1 (127.19.238.67:39859): Not found: Tablet not found: 565bc3b8c685444aaef3ab58bb10e15d
W20260812 16:36:03.933027 6762 leader_election.cc:341] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 15cb6a42b7994dc6ae21bd1d0d3062bb (127.19.238.66:37659): Not found: Tablet not found: 565bc3b8c685444aaef3ab58bb10e15d
I20260812 16:36:03.933064 6765 leader_election.cc:302] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b697ba16edf499e8812522375210c09; no voters: 15cb6a42b7994dc6ae21bd1d0d3062bb, 3f9e8fd85db0448ab43452f87bb4c0e1
I20260812 16:36:03.933136 7223 raft_consensus.cc:2726] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:36:04.317715 7224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:04.317484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:48335)
0812 16:36:04.317630 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:04.317668 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":65,"threads_started":1}
I20260812 16:36:04.362641 7225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:04.362411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:58257)
0812 16:36:04.362551 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:04.362592 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260812 16:36:04.491765 7226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:04.491542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:45691)
0812 16:36:04.491672 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:04.491715 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
I20260812 16:36:05.318575 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:05.318313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:59271)
0812 16:36:05.318485 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:05.318526 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":75,"threads_started":1}
I20260812 16:36:05.363502 7229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:05.363257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:33613)
0812 16:36:05.363406 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:05.363452 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260812 16:36:05.492575 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:05.492321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:44005)
0812 16:36:05.492484 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:05.492527 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":92,"threads_started":1}
I20260812 16:36:06.319420 7232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:06.319143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:56981)
0812 16:36:06.319309 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:06.319372 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
I20260812 16:36:06.364349 7233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:06.364139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:52599)
0812 16:36:06.364266 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:06.364301 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260812 16:36:06.493300 7235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:06.493033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:48673)
0812 16:36:06.493209 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:06.493249 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":81,"threads_started":1}
I20260812 16:36:07.320343 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:07.320057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:50233)
0812 16:36:07.320223 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:07.320283 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":66,"threads_started":1}
I20260812 16:36:07.365186 7238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:07.364903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:46061)
0812 16:36:07.365096 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:07.365138 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":76,"threads_started":1}
I20260812 16:36:07.494498 7239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:07.494211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:36945)
0812 16:36:07.494365 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:07.494410 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":63,"threads_started":1}
I20260812 16:36:08.175793 7240 raft_consensus.cc:482] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:36:08.175969 7240 raft_consensus.cc:504] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9e8fd85db0448ab43452f87bb4c0e1" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39859 } } peers { permanent_uuid: "15cb6a42b7994dc6ae21bd1d0d3062bb" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 37659 } } peers { permanent_uuid: "3b697ba16edf499e8812522375210c09" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 39495 } }
I20260812 16:36:08.176195 7240 leader_election.cc:288] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f9e8fd85db0448ab43452f87bb4c0e1 (127.19.238.67:39859), 15cb6a42b7994dc6ae21bd1d0d3062bb (127.19.238.66:37659)
W20260812 16:36:08.176678 6762 leader_election.cc:341] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 15cb6a42b7994dc6ae21bd1d0d3062bb (127.19.238.66:37659): Not found: Tablet not found: 565bc3b8c685444aaef3ab58bb10e15d
W20260812 16:36:08.176757 6765 leader_election.cc:341] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f9e8fd85db0448ab43452f87bb4c0e1 (127.19.238.67:39859): Not found: Tablet not found: 565bc3b8c685444aaef3ab58bb10e15d
I20260812 16:36:08.176838 6765 leader_election.cc:302] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b697ba16edf499e8812522375210c09; no voters: 15cb6a42b7994dc6ae21bd1d0d3062bb, 3f9e8fd85db0448ab43452f87bb4c0e1
I20260812 16:36:08.176951 7240 raft_consensus.cc:2726] T 565bc3b8c685444aaef3ab58bb10e15d P 3b697ba16edf499e8812522375210c09 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:36:08.321127 7241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:08.320870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:59027)
0812 16:36:08.321030 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:08.321076 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":79,"threads_started":1}
I20260812 16:36:08.366103 7242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:08.365805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:34545)
0812 16:36:08.365994 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:08.366049 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":95,"threads_started":1}
I20260812 16:36:08.495270 7243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:08.495026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:37165)
0812 16:36:08.495182 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:08.495223 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":68,"threads_started":1}
I20260812 16:36:09.321997 7244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:09.321741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.65:59259)
0812 16:36:09.321909 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:09.321950 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":84,"threads_started":1}
I20260812 16:36:09.366914 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:09.366671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.66:46379)
0812 16:36:09.366825 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:09.366866 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":78,"threads_started":1}
I20260812 16:36:09.496095 7246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:09.495879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:39511 (local address 127.19.238.67:35197)
0812 16:36:09.496013 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:09.496050 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.126:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260812 16:36:09.502513 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 6705
I20260812 16:36:09.511472 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 6892
I20260812 16:36:09.519444 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 7044
2026-08-12T16:36:09Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58558 ms)
[----------] 30 tests from SecurityITest (242349 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:09 dist-test-slave-mvvj krb5kdc[7249](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:09 dist-test-slave-mvvj krb5kdc[7249](info): set up 2 sockets
Aug 12 16:36:09 dist-test-slave-mvvj krb5kdc[7249](info): commencing operation
krb5kdc: starting...
W20260812 16:36:10.636749 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:10 dist-test-slave-mvvj krb5kdc[7249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:36:10.776531 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:42887
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:42887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:10.861521 7265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:10.861784 7265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:10.861824 7265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:10.863404 7265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:10.863521 7265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:10.863551 7265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:10.863574 7265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:10.863605 7265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:10.865229 7265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:42887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:42887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:10.865535 7265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:10.865809 7265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:10.871682 7270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:10.871719 7273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:10.871757 7265 server_base.cc:1034] running on GCE node
W20260812 16:36:10.871719 7271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:10.872202 7265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:10.872524 7265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:10.873704 7265 hybrid_clock.cc:648] HybridClock initialized: now 1786552570873702 us; error 41 us; skew 500 ppm
Aug 12 16:36:10 dist-test-slave-mvvj krb5kdc[7249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552570, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:10.876348 7265 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:36:10.876871 7265 webserver.cc:460] Webserver started at http://127.19.238.126:33141/ using document root <none> and password file <none>
I20260812 16:36:10.877117 7265 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:10.877188 7265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:10.877344 7265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:10.878434 7265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "a4731a34722b4e4bb4ac93208f33d49a"
format_stamp: "Formatted at 2026-08-12 16:36:10 on dist-test-slave-mvvj"
server_key: "2b1cc6ea5895c90241e43a1f47918021"
server_key_iv: "69dd548322f7ff22809cad666630cde5"
server_key_version: "encryptionkey@0"
I20260812 16:36:10.878822 7265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "a4731a34722b4e4bb4ac93208f33d49a"
format_stamp: "Formatted at 2026-08-12 16:36:10 on dist-test-slave-mvvj"
server_key: "2b1cc6ea5895c90241e43a1f47918021"
server_key_iv: "69dd548322f7ff22809cad666630cde5"
server_key_version: "encryptionkey@0"
I20260812 16:36:10.880264 7265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:36:10.881035 7281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:10.881228 7265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 16:36:10.881340 7265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "a4731a34722b4e4bb4ac93208f33d49a"
format_stamp: "Formatted at 2026-08-12 16:36:10 on dist-test-slave-mvvj"
server_key: "2b1cc6ea5895c90241e43a1f47918021"
server_key_iv: "69dd548322f7ff22809cad666630cde5"
server_key_version: "encryptionkey@0"
I20260812 16:36:10.881423 7265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:10.890693 7265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:10.892959 7265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:10.893126 7265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:10.897372 7265 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:42887
I20260812 16:36:10.897418 7335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:42887 every 8 connection(s)
I20260812 16:36:10.897800 7265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:10.898727 7336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:10.901108 7336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a: Bootstrap starting.
I20260812 16:36:10.901717 7336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:10.901969 7336 log.cc:824] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:10.902595 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 7265
I20260812 16:36:10.902668 7336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a: No bootstrap required, opened a new log
I20260812 16:36:10.902724 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:10.902926 20409 external_mini_cluster.cc:1428] Setting key 0136ecc072bfe3286bce10356dbbaa0b
I20260812 16:36:10.905504 7336 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4731a34722b4e4bb4ac93208f33d49a" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42887 } }
I20260812 16:36:10.905683 7336 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:10.905723 7336 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4731a34722b4e4bb4ac93208f33d49a, State: Initialized, Role: FOLLOWER
I20260812 16:36:10.905907 7336 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4731a34722b4e4bb4ac93208f33d49a" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42887 } }
I20260812 16:36:10.905995 7336 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:10.906018 7336 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:10.906085 7336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:10.906754 7336 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4731a34722b4e4bb4ac93208f33d49a" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42887 } }
I20260812 16:36:10.906864 7336 leader_election.cc:302] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4731a34722b4e4bb4ac93208f33d49a; no voters:
I20260812 16:36:10.907020 7336 leader_election.cc:288] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:10.907099 7341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:10.907258 7336 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:10.907284 7341 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a [term 1 LEADER]: Becoming Leader. State: Replica: a4731a34722b4e4bb4ac93208f33d49a, State: Running, Role: LEADER
I20260812 16:36:10.907523 7341 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4731a34722b4e4bb4ac93208f33d49a" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42887 } }
I20260812 16:36:10.907970 7343 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4731a34722b4e4bb4ac93208f33d49a. Latest consensus state: current_term: 1 leader_uuid: "a4731a34722b4e4bb4ac93208f33d49a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4731a34722b4e4bb4ac93208f33d49a" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42887 } } }
I20260812 16:36:10.908051 7343 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:10.908382 7342 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4731a34722b4e4bb4ac93208f33d49a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4731a34722b4e4bb4ac93208f33d49a" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 42887 } } }
I20260812 16:36:10.908486 7342 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a [sys.catalog]: This master's current role is: LEADER
W20260812 16:36:10.909157 7360 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:36:10.909210 7360 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:36:10.909255 7351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:10.909462 7351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:10.911178 7351 catalog_manager.cc:1347] Generated new cluster ID: 9f03e6de3f0b4acf914eee24484f20c5
I20260812 16:36:10.911231 7351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:36:10 dist-test-slave-mvvj krb5kdc[7249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:36:10.917053 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:10.903498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36758 (local address 127.19.238.126:42887)
0812 16:36:10.903692 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:10.903702 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:10.903735 (+ 33us) server_negotiation.cc:407] Connection header received
0812 16:36:10.905453 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:10.905464 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:10.905531 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:10.905691 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:10.906453 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:10.907861 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:10.909037 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:10.909638 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:10.909742 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:10.912626 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:10.912630 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:10.912630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:10.912642 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:10.914776 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:10.915218 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:10.915222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:10.915222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:10.915329 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:10.915560 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:10.915561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:10.915562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:10.915926 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:10.916594 (+ 668us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:10.916681 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 16:36:10.916841 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260812 16:36:10.940017 7351 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:10.940665 7351 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:10.948488 7351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a: Generated new TSK 0
I20260812 16:36:10.948693 7351 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
I20260812 16:36:10.987030 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42887
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:11.072566 7366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:11.072774 7366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:11.072799 7366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:11.073937 7366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:11.073973 7366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:11.073998 7366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:36:11.075412 7366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:11.075609 7366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:11.075803 7366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:11.081236 7374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:11.081252 7371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:11.081228 7376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:11.081645 7366 server_base.cc:1034] running on GCE node
I20260812 16:36:11.081863 7366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:11.082106 7366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:11.083289 7366 hybrid_clock.cc:648] HybridClock initialized: now 1786552571083257 us; error 47 us; skew 500 ppm
Aug 12 16:36:11 dist-test-slave-mvvj krb5kdc[7249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552571, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:11.086200 7366 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:36:11.086723 7366 webserver.cc:460] Webserver started at http://127.19.238.65:38345/ using document root <none> and password file <none>
I20260812 16:36:11.086977 7366 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:11.087049 7366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:11.087205 7366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:11.088354 7366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "6183f2ca69054d1fb13aa39959ae016f"
format_stamp: "Formatted at 2026-08-12 16:36:11 on dist-test-slave-mvvj"
server_key: "63e07fa01ce87c4c9cc200e2467363cd"
server_key_iv: "4395a4071bf314efd0ce10df4a257ba9"
server_key_version: "encryptionkey@0"
I20260812 16:36:11.088765 7366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "6183f2ca69054d1fb13aa39959ae016f"
format_stamp: "Formatted at 2026-08-12 16:36:11 on dist-test-slave-mvvj"
server_key: "63e07fa01ce87c4c9cc200e2467363cd"
server_key_iv: "4395a4071bf314efd0ce10df4a257ba9"
server_key_version: "encryptionkey@0"
I20260812 16:36:11.090250 7366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:36:11.091104 7387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:11.091315 7366 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:11.091427 7366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "6183f2ca69054d1fb13aa39959ae016f"
format_stamp: "Formatted at 2026-08-12 16:36:11 on dist-test-slave-mvvj"
server_key: "63e07fa01ce87c4c9cc200e2467363cd"
server_key_iv: "4395a4071bf314efd0ce10df4a257ba9"
server_key_version: "encryptionkey@0"
I20260812 16:36:11.091538 7366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:11.104701 7366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:11.106724 7366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:11.106899 7366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:11.107147 7366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:36:11.107524 7366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:36:11.107579 7366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:11.107625 7366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:36:11.107652 7366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:11.114130 7366 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:40761
I20260812 16:36:11.114172 7512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:40761 every 8 connection(s)
I20260812 16:36:11.114521 7366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
Aug 12 16:36:11 dist-test-slave-mvvj krb5kdc[7249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552571, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:36:11.123168 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 7366
I20260812 16:36:11.123296 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:36:11.123523 20409 external_mini_cluster.cc:1428] Setting key 49ca558a36c25666b6e82ac86c5949e7
I20260812 16:36:11.124158 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.115305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:49275 (local address 127.19.238.126:42887)
0812 16:36:11.115394 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:11.115397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:11.115605 (+ 208us) server_negotiation.cc:407] Connection header received
0812 16:36:11.116213 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:11.116214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:11.116226 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:11.116281 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:11.117188 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.117703 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.118688 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.118891 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.118946 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.121120 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:11.121121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:11.121122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:11.121128 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:11.122531 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:11.122922 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:11.122923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:11.122924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:11.122981 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:11.123304 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:11.123306 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:11.123306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:11.123495 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:11.123558 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:11.123954 (+ 396us) server_negotiation.cc:299] Negotiation successful
0812 16:36:11.124062 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 16:36:11.124323 7515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.115256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:42887 (local address 127.19.238.65:49275)
0812 16:36:11.115477 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:11.115515 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:11.116105 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:11.116333 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:11.116337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:11.116902 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:11.117098 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:11.117103 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.117812 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.117812 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:11.118609 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:11.118611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.118978 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.118979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:11.119089 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.119541 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:11.119549 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:11.121014 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:11.122608 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:11.122615 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:11.122617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:11.122838 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:11.123053 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:11.123054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:11.123055 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:11.123206 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:11.123576 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:11.123584 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:11.123758 (+ 174us) client_negotiation.cc:769] Sending connection context
0812 16:36:11.123924 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 16:36:11.124069 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":108,"threads_started":1}
I20260812 16:36:11.124933 7513 heartbeater.cc:347] Connected to a master server at 127.19.238.126:42887
I20260812 16:36:11.125029 7513 heartbeater.cc:464] Registering TS with master...
I20260812 16:36:11.125226 7513 heartbeater.cc:511] Master 127.19.238.126:42887 requested a full tablet report, sending...
I20260812 16:36:11.125804 7299 ts_manager.cc:194] Registered new tserver with Master: 6183f2ca69054d1fb13aa39959ae016f (127.19.238.65:40761)
I20260812 16:36:11.126566 7299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:49275
WARNING: no policy specified for kudu/127.19.238.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
I20260812 16:36:11.151152 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42887
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:11.235328 7518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:11.235538 7518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:11.235563 7518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:11.236804 7518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:11.236851 7518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:11.236877 7518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:36:11.238438 7518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:11.238710 7518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:11.238961 7518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:11.244423 7527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:11.244617 7518 server_base.cc:1034] running on GCE node
W20260812 16:36:11.244467 7524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:11.244423 7525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:11.244942 7518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:11.245165 7518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:11.246309 7518 hybrid_clock.cc:648] HybridClock initialized: now 1786552571246289 us; error 36 us; skew 500 ppm
Aug 12 16:36:11 dist-test-slave-mvvj krb5kdc[7249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552571, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:11.248934 7518 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:36:11.249428 7518 webserver.cc:460] Webserver started at http://127.19.238.66:38617/ using document root <none> and password file <none>
I20260812 16:36:11.249644 7518 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:11.249686 7518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:11.249778 7518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:11.250851 7518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "4a4889f0446f4efab1041d94987332ff"
format_stamp: "Formatted at 2026-08-12 16:36:11 on dist-test-slave-mvvj"
server_key: "0ded14b16c76a0da49ffc589c030ef4f"
server_key_iv: "6eac62d24cdc18f7e056cf42f4ade626"
server_key_version: "encryptionkey@0"
I20260812 16:36:11.251156 7518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "4a4889f0446f4efab1041d94987332ff"
format_stamp: "Formatted at 2026-08-12 16:36:11 on dist-test-slave-mvvj"
server_key: "0ded14b16c76a0da49ffc589c030ef4f"
server_key_iv: "6eac62d24cdc18f7e056cf42f4ade626"
server_key_version: "encryptionkey@0"
I20260812 16:36:11.252573 7518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 16:36:11.253291 7534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:11.253468 7518 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:36:11.253602 7518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "4a4889f0446f4efab1041d94987332ff"
format_stamp: "Formatted at 2026-08-12 16:36:11 on dist-test-slave-mvvj"
server_key: "0ded14b16c76a0da49ffc589c030ef4f"
server_key_iv: "6eac62d24cdc18f7e056cf42f4ade626"
server_key_version: "encryptionkey@0"
I20260812 16:36:11.253685 7518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:11.271951 7518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:11.274471 7518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:11.274650 7518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:11.274874 7518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:36:11.275209 7518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:36:11.275266 7518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:11.275314 7518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:36:11.275338 7518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:11.281436 7518 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:36869
I20260812 16:36:11.281517 7665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:36869 every 8 connection(s)
I20260812 16:36:11.281872 7518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:36:11.287014 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 7518
I20260812 16:36:11.287158 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:36:11.287415 20409 external_mini_cluster.cc:1428] Setting key 27c73e9b465c8af063d5efa3ea1ac565
Aug 12 16:36:11 dist-test-slave-mvvj krb5kdc[7249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552571, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:36:11.292393 7668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.282663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:42887 (local address 127.19.238.66:60907)
0812 16:36:11.282868 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:11.282906 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:11.283454 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:11.283722 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:11.283731 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:11.284470 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:11.284686 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:11.284692 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.285428 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.285429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:11.286265 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:11.286268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.286665 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.286666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:11.286802 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.287434 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:11.287443 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:11.289096 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:11.290841 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:11.290849 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:11.290852 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:11.291185 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:11.291418 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:11.291418 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:11.291419 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:11.291526 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:11.291821 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:11.291827 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:11.292012 (+ 185us) client_negotiation.cc:769] Sending connection context
0812 16:36:11.292103 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 16:36:11.292226 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":99,"threads_started":1}
I20260812 16:36:11.292388 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.282714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:60907 (local address 127.19.238.126:42887)
0812 16:36:11.282801 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:11.282805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:11.282991 (+ 186us) server_negotiation.cc:407] Connection header received
0812 16:36:11.283574 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:11.283575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:11.283588 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:11.283649 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:11.284798 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.285325 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.286366 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.286574 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.286653 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.289189 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:11.289191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:11.289191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:11.289201 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:11.290752 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:11.291283 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:11.291285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:11.291285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:11.291350 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:11.291594 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:11.291595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:11.291595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:11.291750 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:11.291823 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:11.292152 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 16:36:11.292247 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 16:36:11.293272 7666 heartbeater.cc:347] Connected to a master server at 127.19.238.126:42887
I20260812 16:36:11.293388 7666 heartbeater.cc:464] Registering TS with master...
I20260812 16:36:11.293607 7666 heartbeater.cc:511] Master 127.19.238.126:42887 requested a full tablet report, sending...
I20260812 16:36:11.294013 7299 ts_manager.cc:194] Registered new tserver with Master: 4a4889f0446f4efab1041d94987332ff (127.19.238.66:36869)
I20260812 16:36:11.294472 7299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:60907
WARNING: no policy specified for kudu/127.19.238.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
I20260812 16:36:11.316857 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42887
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:11.401105 7671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:11.401316 7671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:11.401341 7671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:11.402622 7671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:11.402670 7671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:11.402697 7671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:36:11.404212 7671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:42887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:11.404474 7671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:11.404712 7671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:11.410022 7676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:11.410099 7671 server_base.cc:1034] running on GCE node
W20260812 16:36:11.410025 7680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:11.410086 7677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:11.410493 7671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:11.410751 7671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:11.411895 7671 hybrid_clock.cc:648] HybridClock initialized: now 1786552571411867 us; error 58 us; skew 500 ppm
Aug 12 16:36:11 dist-test-slave-mvvj krb5kdc[7249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552571, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:11.414480 7671 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:36:11.414985 7671 webserver.cc:460] Webserver started at http://127.19.238.67:34357/ using document root <none> and password file <none>
I20260812 16:36:11.415238 7671 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:11.415309 7671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:11.415452 7671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:11.416749 7671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "a9455ca86fff4229abbe278d848d0e36"
format_stamp: "Formatted at 2026-08-12 16:36:11 on dist-test-slave-mvvj"
server_key: "e5e15297899d5a8cc2c95e403c13cb9c"
server_key_iv: "b2065b9ba9a44ab734518732dd0abb15"
server_key_version: "encryptionkey@0"
I20260812 16:36:11.417176 7671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "a9455ca86fff4229abbe278d848d0e36"
format_stamp: "Formatted at 2026-08-12 16:36:11 on dist-test-slave-mvvj"
server_key: "e5e15297899d5a8cc2c95e403c13cb9c"
server_key_iv: "b2065b9ba9a44ab734518732dd0abb15"
server_key_version: "encryptionkey@0"
I20260812 16:36:11.418604 7671 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:36:11.419417 7688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:11.419646 7671 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:11.419759 7671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "a9455ca86fff4229abbe278d848d0e36"
format_stamp: "Formatted at 2026-08-12 16:36:11 on dist-test-slave-mvvj"
server_key: "e5e15297899d5a8cc2c95e403c13cb9c"
server_key_iv: "b2065b9ba9a44ab734518732dd0abb15"
server_key_version: "encryptionkey@0"
I20260812 16:36:11.419863 7671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:11.433478 7671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:11.435313 7671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:11.435480 7671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:11.435734 7671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:36:11.436100 7671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:36:11.436159 7671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:11.436203 7671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:36:11.436232 7671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:11.442678 7671 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:41625
I20260812 16:36:11.442732 7818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:41625 every 8 connection(s)
I20260812 16:36:11.443060 7671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
Aug 12 16:36:11 dist-test-slave-mvvj krb5kdc[7249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552571, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:36:11.452445 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 7671
I20260812 16:36:11.452600 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:36:11.452821 20409 external_mini_cluster.cc:1428] Setting key cfcb78bda3b770a6e8e3746a1639e1b6
I20260812 16:36:11.453105 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.443753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:36977 (local address 127.19.238.126:42887)
0812 16:36:11.443801 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:11.443803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:11.444051 (+ 248us) server_negotiation.cc:407] Connection header received
0812 16:36:11.444811 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:11.444813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:11.444829 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:11.444865 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:11.446029 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.446561 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.447510 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.447693 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.447745 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.450242 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:11.450243 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:11.450244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:11.450250 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:11.451462 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:11.451915 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:11.451916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:11.451917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:11.451979 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:11.452223 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:11.452224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:11.452224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:11.452429 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:11.452493 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:11.452856 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 16:36:11.452970 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 16:36:11.453263 7821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.443739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:42887 (local address 127.19.238.67:36977)
0812 16:36:11.443940 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:11.443976 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:11.444666 (+ 690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:11.445061 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:11.445065 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:11.445734 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:11.445937 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:11.445941 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.446652 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.446652 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:11.447420 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:11.447422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.447768 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.447768 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:11.447892 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.448582 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:11.448593 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:11.450133 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:11.451535 (+ 1402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:11.451555 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:11.451557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:11.451765 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:11.452057 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:11.452058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:11.452058 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:11.452157 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:11.452511 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:11.452518 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:11.452694 (+ 176us) client_negotiation.cc:769] Sending connection context
0812 16:36:11.452825 (+ 131us) client_negotiation.cc:240] Negotiation successful
0812 16:36:11.452996 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":123,"threads_started":1}
I20260812 16:36:11.453933 7819 heartbeater.cc:347] Connected to a master server at 127.19.238.126:42887
I20260812 16:36:11.454017 7819 heartbeater.cc:464] Registering TS with master...
I20260812 16:36:11.454154 7819 heartbeater.cc:511] Master 127.19.238.126:42887 requested a full tablet report, sending...
I20260812 16:36:11.454489 7298 ts_manager.cc:194] Registered new tserver with Master: a9455ca86fff4229abbe278d848d0e36 (127.19.238.67:41625)
I20260812 16:36:11.454872 7298 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:36977
I20260812 16:36:11.464926 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:36:11.473183 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.466709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36768 (local address 127.19.238.126:42887)
0812 16:36:11.466934 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:11.466936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:11.466948 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:36:11.466977 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:11.466978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:11.466989 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:11.467041 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:11.467657 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.468369 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.469141 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.469359 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.469423 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.470642 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:11.470643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:11.470644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:11.470650 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:11.472156 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:11.472497 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:11.472498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:11.472498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:11.472550 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:11.472719 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:11.472719 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:11.472720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:11.472868 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:11.472944 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:11.473031 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 16:36:11.473090 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196}
I20260812 16:36:11.474962 7298 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:36:11.476678 7298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:36:11.483412 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.479292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:36869 (local address 127.0.0.1:41420)
0812 16:36:11.479629 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:11.479638 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:11.479691 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:11.480064 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:11.480065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:11.480068 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:11.480111 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:11.480113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.481366 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.481367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:11.482139 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:11.482141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.483077 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.483078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:11.483156 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.483171 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 16:36:11.483246 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 16:36:11.483297 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":83,"threads_started":1}
I20260812 16:36:11.483520 7836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.479453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50090 (local address 127.19.238.65:40761)
0812 16:36:11.479866 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:11.479870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:11.479880 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:36:11.479922 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:11.479924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:11.479940 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:11.480015 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:11.480251 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.480959 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.482422 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.483150 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.483245 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.483302 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:11.483394 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 16:36:11.483435 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":90,"threads_started":1}
I20260812 16:36:11.483637 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.479402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41420 (local address 127.19.238.66:36869)
0812 16:36:11.479618 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:11.479622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:11.479684 (+ 62us) server_negotiation.cc:407] Connection header received
0812 16:36:11.479757 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:11.479760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:11.479774 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:11.479975 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:11.480191 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.481257 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.482234 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.482985 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.483067 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.483128 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:11.483483 (+ 355us) server_negotiation.cc:299] Negotiation successful
0812 16:36:11.483531 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":115,"threads_started":1}
I20260812 16:36:11.483800 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.479148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:40761 (local address 127.0.0.1:50090)
0812 16:36:11.479616 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:11.479628 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:11.479687 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:11.480105 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:11.480106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:11.480108 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:11.480162 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:11.480164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.481071 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.481072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:11.482333 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:11.482336 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.483248 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.483249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:11.483315 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.483319 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:36:11.483378 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 16:36:11.483421 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":45,"thread_start_us":99,"threads_started":1}
I20260812 16:36:11.483932 7833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.479241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39124 (local address 127.19.238.67:41625)
0812 16:36:11.479431 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:11.479436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:11.479664 (+ 228us) server_negotiation.cc:407] Connection header received
0812 16:36:11.479711 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:11.479713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:11.479732 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:11.479805 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:11.480076 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.481234 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.482720 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.483328 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.483389 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.483441 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:11.483775 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 16:36:11.483849 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
I20260812 16:36:11.484037 7830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.479077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:41625 (local address 127.0.0.1:39124)
0812 16:36:11.479436 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:11.479479 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:11.479571 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:11.479915 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:11.479919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:11.479921 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:11.479994 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:11.479996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.481355 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.481356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:11.482641 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:11.482642 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.483476 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.483477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:11.483555 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.483559 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:36:11.483614 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 16:36:11.483653 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":8,"thread_start_us":72,"threads_started":1}
I20260812 16:36:11.484637 7592 tablet_service.cc:1467] Processing CreateTablet for tablet d62f66e9318645d2ae95afb298dbd4aa (DEFAULT_TABLE table=test-table [id=af593dcf965e48e4aa00c66b34654951]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:36:11.484676 7443 tablet_service.cc:1467] Processing CreateTablet for tablet d62f66e9318645d2ae95afb298dbd4aa (DEFAULT_TABLE table=test-table [id=af593dcf965e48e4aa00c66b34654951]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:36:11.485013 7592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d62f66e9318645d2ae95afb298dbd4aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:11.485044 7443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d62f66e9318645d2ae95afb298dbd4aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:11.487962 7837 tablet_bootstrap.cc:492] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff: Bootstrap starting.
I20260812 16:36:11.487895 7746 tablet_service.cc:1467] Processing CreateTablet for tablet d62f66e9318645d2ae95afb298dbd4aa (DEFAULT_TABLE table=test-table [id=af593dcf965e48e4aa00c66b34654951]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:36:11.488263 7746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d62f66e9318645d2ae95afb298dbd4aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:11.488343 7838 tablet_bootstrap.cc:492] T d62f66e9318645d2ae95afb298dbd4aa P 6183f2ca69054d1fb13aa39959ae016f: Bootstrap starting.
I20260812 16:36:11.488739 7837 tablet_bootstrap.cc:654] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:11.488968 7838 tablet_bootstrap.cc:654] T d62f66e9318645d2ae95afb298dbd4aa P 6183f2ca69054d1fb13aa39959ae016f: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:11.489343 7837 log.cc:824] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:11.489594 7838 log.cc:824] T d62f66e9318645d2ae95afb298dbd4aa P 6183f2ca69054d1fb13aa39959ae016f: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:11.490231 7837 tablet_bootstrap.cc:492] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff: No bootstrap required, opened a new log
I20260812 16:36:11.490310 7837 ts_tablet_manager.cc:1397] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 16:36:11.490373 7838 tablet_bootstrap.cc:492] T d62f66e9318645d2ae95afb298dbd4aa P 6183f2ca69054d1fb13aa39959ae016f: No bootstrap required, opened a new log
I20260812 16:36:11.490444 7838 ts_tablet_manager.cc:1397] T d62f66e9318645d2ae95afb298dbd4aa P 6183f2ca69054d1fb13aa39959ae016f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:36:11.491870 7842 tablet_bootstrap.cc:492] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36: Bootstrap starting.
I20260812 16:36:11.491961 7838 raft_consensus.cc:348] T d62f66e9318645d2ae95afb298dbd4aa P 6183f2ca69054d1fb13aa39959ae016f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9455ca86fff4229abbe278d848d0e36" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 41625 } } peers { permanent_uuid: "4a4889f0446f4efab1041d94987332ff" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36869 } } peers { permanent_uuid: "6183f2ca69054d1fb13aa39959ae016f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 40761 } }
I20260812 16:36:11.492262 7838 raft_consensus.cc:374] T d62f66e9318645d2ae95afb298dbd4aa P 6183f2ca69054d1fb13aa39959ae016f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:11.492363 7838 raft_consensus.cc:729] T d62f66e9318645d2ae95afb298dbd4aa P 6183f2ca69054d1fb13aa39959ae016f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6183f2ca69054d1fb13aa39959ae016f, State: Initialized, Role: FOLLOWER
I20260812 16:36:11.492591 7842 tablet_bootstrap.cc:654] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:11.492558 7838 consensus_queue.cc:260] T d62f66e9318645d2ae95afb298dbd4aa P 6183f2ca69054d1fb13aa39959ae016f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9455ca86fff4229abbe278d848d0e36" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 41625 } } peers { permanent_uuid: "4a4889f0446f4efab1041d94987332ff" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36869 } } peers { permanent_uuid: "6183f2ca69054d1fb13aa39959ae016f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 40761 } }
I20260812 16:36:11.492491 7837 raft_consensus.cc:348] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9455ca86fff4229abbe278d848d0e36" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 41625 } } peers { permanent_uuid: "4a4889f0446f4efab1041d94987332ff" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36869 } } peers { permanent_uuid: "6183f2ca69054d1fb13aa39959ae016f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 40761 } }
I20260812 16:36:11.492733 7837 raft_consensus.cc:374] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:11.492785 7837 raft_consensus.cc:729] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a4889f0446f4efab1041d94987332ff, State: Initialized, Role: FOLLOWER
I20260812 16:36:11.492892 7838 ts_tablet_manager.cc:1428] T d62f66e9318645d2ae95afb298dbd4aa P 6183f2ca69054d1fb13aa39959ae016f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 16:36:11.492916 7837 consensus_queue.cc:260] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9455ca86fff4229abbe278d848d0e36" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 41625 } } peers { permanent_uuid: "4a4889f0446f4efab1041d94987332ff" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36869 } } peers { permanent_uuid: "6183f2ca69054d1fb13aa39959ae016f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 40761 } }
I20260812 16:36:11.493088 7513 heartbeater.cc:503] Master 127.19.238.126:42887 was elected leader, sending a full tablet report...
I20260812 16:36:11.493220 7666 heartbeater.cc:503] Master 127.19.238.126:42887 was elected leader, sending a full tablet report...
I20260812 16:36:11.493228 7842 log.cc:824] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:11.493207 7837 ts_tablet_manager.cc:1428] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:36:11.494179 7842 tablet_bootstrap.cc:492] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36: No bootstrap required, opened a new log
I20260812 16:36:11.494270 7842 ts_tablet_manager.cc:1397] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:36:11.495725 7842 raft_consensus.cc:348] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9455ca86fff4229abbe278d848d0e36" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 41625 } } peers { permanent_uuid: "4a4889f0446f4efab1041d94987332ff" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36869 } } peers { permanent_uuid: "6183f2ca69054d1fb13aa39959ae016f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 40761 } }
I20260812 16:36:11.495921 7842 raft_consensus.cc:374] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:11.495960 7842 raft_consensus.cc:729] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9455ca86fff4229abbe278d848d0e36, State: Initialized, Role: FOLLOWER
I20260812 16:36:11.496066 7842 consensus_queue.cc:260] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9455ca86fff4229abbe278d848d0e36" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 41625 } } peers { permanent_uuid: "4a4889f0446f4efab1041d94987332ff" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36869 } } peers { permanent_uuid: "6183f2ca69054d1fb13aa39959ae016f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 40761 } }
I20260812 16:36:11.496356 7842 ts_tablet_manager.cc:1428] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:36:11.496459 7819 heartbeater.cc:503] Master 127.19.238.126:42887 was elected leader, sending a full tablet report...
W20260812 16:36:11.532464 7667 tablet.cc:2369] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:36:11.546075 7846 raft_consensus.cc:482] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:36:11.546234 7846 raft_consensus.cc:504] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9455ca86fff4229abbe278d848d0e36" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 41625 } } peers { permanent_uuid: "4a4889f0446f4efab1041d94987332ff" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36869 } } peers { permanent_uuid: "6183f2ca69054d1fb13aa39959ae016f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 40761 } }
I20260812 16:36:11.546669 7846 leader_election.cc:288] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a4889f0446f4efab1041d94987332ff (127.19.238.66:36869), 6183f2ca69054d1fb13aa39959ae016f (127.19.238.65:40761)
I20260812 16:36:11.550666 7836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.547010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:40397 (local address 127.19.238.65:40761)
0812 16:36:11.547053 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:11.547055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:11.547355 (+ 300us) server_negotiation.cc:407] Connection header received
0812 16:36:11.547389 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:11.547390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:11.547404 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:11.547458 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:11.547668 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.548325 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.549621 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.550220 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.550306 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.550315 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:11.550563 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 16:36:11.550605 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 16:36:11.550659 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.546929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:37907 (local address 127.19.238.66:36869)
0812 16:36:11.546982 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:11.546984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:11.547017 (+ 33us) server_negotiation.cc:407] Connection header received
0812 16:36:11.547070 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:11.547071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:11.547083 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:11.547171 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:11.547342 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.548126 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.549621 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.550231 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.550306 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.550315 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:11.550540 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 16:36:11.550587 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 16:36:11.550652 7821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.546844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:36869 (local address 127.19.238.67:37907)
0812 16:36:11.546894 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:11.546909 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:11.546964 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:11.547178 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:11.547179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:11.547181 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:11.547262 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:11.547264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.548226 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.548227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:11.549506 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:11.549508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.550334 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.550335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:11.550431 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.550435 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:36:11.550500 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 16:36:11.550551 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":55}
I20260812 16:36:11.550661 7847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.546880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:40761 (local address 127.19.238.67:40397)
0812 16:36:11.547074 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:11.547084 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:11.547134 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:11.547507 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:11.547508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:11.547510 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:11.547581 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:11.547583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.548427 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.548428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:11.549509 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:11.549510 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.550334 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.550335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:11.550430 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.550435 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:36:11.550532 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 16:36:11.550560 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":3,"thread_start_us":127,"threads_started":1}
I20260812 16:36:11.551116 7464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d62f66e9318645d2ae95afb298dbd4aa" candidate_uuid: "a9455ca86fff4229abbe278d848d0e36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183f2ca69054d1fb13aa39959ae016f" is_pre_election: true
I20260812 16:36:11.551117 7616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d62f66e9318645d2ae95afb298dbd4aa" candidate_uuid: "a9455ca86fff4229abbe278d848d0e36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a4889f0446f4efab1041d94987332ff" is_pre_election: true
I20260812 16:36:11.551306 7464 raft_consensus.cc:2454] T d62f66e9318645d2ae95afb298dbd4aa P 6183f2ca69054d1fb13aa39959ae016f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9455ca86fff4229abbe278d848d0e36 in term 0.
I20260812 16:36:11.551318 7616 raft_consensus.cc:2454] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9455ca86fff4229abbe278d848d0e36 in term 0.
I20260812 16:36:11.551573 7689 leader_election.cc:302] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6183f2ca69054d1fb13aa39959ae016f, a9455ca86fff4229abbe278d848d0e36; no voters:
I20260812 16:36:11.551713 7846 raft_consensus.cc:2781] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:36:11.551771 7846 raft_consensus.cc:482] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:36:11.551798 7846 raft_consensus.cc:3037] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:11.552568 7846 raft_consensus.cc:504] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9455ca86fff4229abbe278d848d0e36" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 41625 } } peers { permanent_uuid: "4a4889f0446f4efab1041d94987332ff" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36869 } } peers { permanent_uuid: "6183f2ca69054d1fb13aa39959ae016f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 40761 } }
I20260812 16:36:11.552716 7846 leader_election.cc:288] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [CANDIDATE]: Term 1 election: Requested vote from peers 4a4889f0446f4efab1041d94987332ff (127.19.238.66:36869), 6183f2ca69054d1fb13aa39959ae016f (127.19.238.65:40761)
I20260812 16:36:11.552875 7464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d62f66e9318645d2ae95afb298dbd4aa" candidate_uuid: "a9455ca86fff4229abbe278d848d0e36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183f2ca69054d1fb13aa39959ae016f"
I20260812 16:36:11.552940 7616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d62f66e9318645d2ae95afb298dbd4aa" candidate_uuid: "a9455ca86fff4229abbe278d848d0e36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a4889f0446f4efab1041d94987332ff"
I20260812 16:36:11.552985 7464 raft_consensus.cc:3037] T d62f66e9318645d2ae95afb298dbd4aa P 6183f2ca69054d1fb13aa39959ae016f [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:11.553026 7616 raft_consensus.cc:3037] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:11.553900 7616 raft_consensus.cc:2454] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9455ca86fff4229abbe278d848d0e36 in term 1.
I20260812 16:36:11.553996 7464 raft_consensus.cc:2454] T d62f66e9318645d2ae95afb298dbd4aa P 6183f2ca69054d1fb13aa39959ae016f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9455ca86fff4229abbe278d848d0e36 in term 1.
I20260812 16:36:11.554069 7690 leader_election.cc:302] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a4889f0446f4efab1041d94987332ff, a9455ca86fff4229abbe278d848d0e36; no voters:
I20260812 16:36:11.554171 7846 raft_consensus.cc:2781] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:11.554320 7846 raft_consensus.cc:686] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [term 1 LEADER]: Becoming Leader. State: Replica: a9455ca86fff4229abbe278d848d0e36, State: Running, Role: LEADER
I20260812 16:36:11.554388 7846 consensus_queue.cc:237] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9455ca86fff4229abbe278d848d0e36" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 41625 } } peers { permanent_uuid: "4a4889f0446f4efab1041d94987332ff" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36869 } } peers { permanent_uuid: "6183f2ca69054d1fb13aa39959ae016f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 40761 } }
I20260812 16:36:11.555275 7299 catalog_manager.cc:5495] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9455ca86fff4229abbe278d848d0e36 (127.19.238.67). New cstate: current_term: 1 leader_uuid: "a9455ca86fff4229abbe278d848d0e36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9455ca86fff4229abbe278d848d0e36" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 41625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a4889f0446f4efab1041d94987332ff" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6183f2ca69054d1fb13aa39959ae016f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 40761 } health_report { overall_health: UNKNOWN } } }
I20260812 16:36:11.577387 7833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.575056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39140 (local address 127.19.238.67:41625)
0812 16:36:11.575130 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:11.575132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:11.575142 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:36:11.575183 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:11.575184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:11.575203 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:11.575270 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:36:11.575411 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.575906 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.576624 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.576816 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.576892 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.577044 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:36:11.577144 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:36:11.577204 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:11.577238 (+ 34us) server_negotiation.cc:299] Negotiation successful
0812 16:36:11.577256 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 16:36:11.579409 7464 raft_consensus.cc:1261] T d62f66e9318645d2ae95afb298dbd4aa P 6183f2ca69054d1fb13aa39959ae016f [term 1 FOLLOWER]: Refusing update from remote peer a9455ca86fff4229abbe278d848d0e36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:36:11.579409 7616 raft_consensus.cc:1261] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff [term 1 FOLLOWER]: Refusing update from remote peer a9455ca86fff4229abbe278d848d0e36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:36:11.579805 7848 consensus_queue.cc:1035] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6183f2ca69054d1fb13aa39959ae016f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:36:11.579999 7846 consensus_queue.cc:1035] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a4889f0446f4efab1041d94987332ff" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 36869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:36:11.582330 7851 mvcc.cc:205] Tried to move back new op lower bound from 7317719333186097152 to 7317719333087436800. Current Snapshot: MvccSnapshot[applied={T|T < 7317719333186097152}]
I20260812 16:36:11.584726 7299 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:36:11.584825 7299 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:36:11.585613 7299 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a4731a34722b4e4bb4ac93208f33d49a: Sending DeleteTablet for 3 replicas of tablet d62f66e9318645d2ae95afb298dbd4aa
I20260812 16:36:11.585928 7746 tablet_service.cc:1514] Processing DeleteTablet for tablet d62f66e9318645d2ae95afb298dbd4aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:36:11 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:39124
I20260812 16:36:11.585973 7443 tablet_service.cc:1514] Processing DeleteTablet for tablet d62f66e9318645d2ae95afb298dbd4aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:36:11 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:50090
I20260812 16:36:11.586153 7862 tablet_replica.cc:330] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36: stopping tablet replica
I20260812 16:36:11.586239 7862 raft_consensus.cc:2229] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:36:11.586236 7863 tablet_replica.cc:330] T d62f66e9318645d2ae95afb298dbd4aa P 6183f2ca69054d1fb13aa39959ae016f: stopping tablet replica
I20260812 16:36:11.586238 7592 tablet_service.cc:1514] Processing DeleteTablet for tablet d62f66e9318645d2ae95afb298dbd4aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:36:11 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:41420
I20260812 16:36:11.586324 7862 raft_consensus.cc:2258] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:36:11.586334 7863 raft_consensus.cc:2229] T d62f66e9318645d2ae95afb298dbd4aa P 6183f2ca69054d1fb13aa39959ae016f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:36:11.586439 7863 raft_consensus.cc:2258] T d62f66e9318645d2ae95afb298dbd4aa P 6183f2ca69054d1fb13aa39959ae016f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:36:11.586910 7863 ts_tablet_manager.cc:1905] T d62f66e9318645d2ae95afb298dbd4aa P 6183f2ca69054d1fb13aa39959ae016f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:36:11.586908 7862 ts_tablet_manager.cc:1905] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:36:11.587178 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 7366
I20260812 16:36:11.590726 7864 tablet_replica.cc:330] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff: stopping tablet replica
I20260812 16:36:11.590914 7864 raft_consensus.cc:2229] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:36:11.590953 7862 ts_tablet_manager.cc:1918] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:36:11.591058 7862 log.cc:1196] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/d62f66e9318645d2ae95afb298dbd4aa
I20260812 16:36:11.591060 7864 raft_consensus.cc:2258] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:36:11.591403 7862 ts_tablet_manager.cc:1939] T d62f66e9318645d2ae95afb298dbd4aa P a9455ca86fff4229abbe278d848d0e36: Deleting consensus metadata
I20260812 16:36:11.591511 7864 ts_tablet_manager.cc:1905] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:36:11.591876 7283 catalog_manager.cc:4841] TS a9455ca86fff4229abbe278d848d0e36 (127.19.238.67:41625): tablet d62f66e9318645d2ae95afb298dbd4aa (table test-table [id=af593dcf965e48e4aa00c66b34654951]) successfully deleted
I20260812 16:36:11.593477 7864 ts_tablet_manager.cc:1918] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:36:11.593557 7864 log.cc:1196] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/d62f66e9318645d2ae95afb298dbd4aa
I20260812 16:36:11.593829 7864 ts_tablet_manager.cc:1939] T d62f66e9318645d2ae95afb298dbd4aa P 4a4889f0446f4efab1041d94987332ff: Deleting consensus metadata
I20260812 16:36:11.594318 7283 catalog_manager.cc:4841] TS 4a4889f0446f4efab1041d94987332ff (127.19.238.66:36869): tablet d62f66e9318645d2ae95afb298dbd4aa (table test-table [id=af593dcf965e48e4aa00c66b34654951]) successfully deleted
W20260812 16:36:11.596726 7282 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.238.65:40761 (error 108)
I20260812 16:36:11.597262 7830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.597122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:40761 (local address 127.0.0.1:50106)
0812 16:36:11.597182 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:11.597222 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:40761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 16:36:11.597379 7282 catalog_manager.cc:4584] TS 6183f2ca69054d1fb13aa39959ae016f (127.19.238.65:40761): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d62f66e9318645d2ae95afb298dbd4aa: Network error: Client connection negotiation failed: client connection to 127.19.238.65:40761: connect: Connection refused (error 111)
I20260812 16:36:11.597711 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 7518
I20260812 16:36:11.609237 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 7671
I20260812 16:36:11.616423 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 7265
2026-08-12T16:36:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2094 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T16:36:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:11Z Disabled control of system clock
I20260812 16:36:11.645701 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:35537
--webserver_interface=127.19.238.126
--webserver_port=0
--builtin_ntp_servers=127.19.238.84:38725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:35537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:36:11.730080 7870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:11.730307 7870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:11.730329 7870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:11.731432 7870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:11.731478 7870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:11.731491 7870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:11.731503 7870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:11.731515 7870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:11.732865 7870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:38725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:35537
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:35537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:11.733067 7870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:11.733330 7870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:11.738502 7876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:11.738513 7877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:11.738524 7879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:11.738850 7870 server_base.cc:1034] running on GCE node
I20260812 16:36:11.739156 7870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:11.739480 7870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:11.740696 7870 hybrid_clock.cc:648] HybridClock initialized: now 1786552571740665 us; error 45 us; skew 500 ppm
I20260812 16:36:11.742304 7870 webserver.cc:460] Webserver started at http://127.19.238.126:34033/ using document root <none> and password file <none>
I20260812 16:36:11.742651 7870 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:11.742745 7870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:11.742974 7870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:11.744311 7870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "87e03002d1ba40269cb3940684ea7296"
format_stamp: "Formatted at 2026-08-12 16:36:11 on dist-test-slave-mvvj"
server_key: "63655601c95054c873d72706203bd92e"
server_key_iv: "8b64b32e04edf6d4cb13811a1bddd639"
server_key_version: "encryptionkey@0"
I20260812 16:36:11.744722 7870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "87e03002d1ba40269cb3940684ea7296"
format_stamp: "Formatted at 2026-08-12 16:36:11 on dist-test-slave-mvvj"
server_key: "63655601c95054c873d72706203bd92e"
server_key_iv: "8b64b32e04edf6d4cb13811a1bddd639"
server_key_version: "encryptionkey@0"
I20260812 16:36:11.746260 7870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:36:11.747169 7885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:11.747390 7870 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:11.747514 7870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "87e03002d1ba40269cb3940684ea7296"
format_stamp: "Formatted at 2026-08-12 16:36:11 on dist-test-slave-mvvj"
server_key: "63655601c95054c873d72706203bd92e"
server_key_iv: "8b64b32e04edf6d4cb13811a1bddd639"
server_key_version: "encryptionkey@0"
I20260812 16:36:11.747601 7870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:11.774060 7870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:11.774430 7870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:11.774602 7870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:11.778753 7870 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:35537
I20260812 16:36:11.778793 7944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:35537 every 8 connection(s)
I20260812 16:36:11.779165 7870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:11.780248 7945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:11.780674 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 7870
I20260812 16:36:11.780794 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:11.781008 20409 external_mini_cluster.cc:1428] Setting key 494f7c2be37a7ee259fd0d2c0a11f304
I20260812 16:36:11.783336 7945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296: Bootstrap starting.
I20260812 16:36:11.784099 7945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:11.784416 7945 log.cc:824] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:11.785224 7945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296: No bootstrap required, opened a new log
I20260812 16:36:11.786947 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.781640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36000 (local address 127.19.238.126:35537)
0812 16:36:11.781837 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:11.781844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:11.781869 (+ 25us) server_negotiation.cc:407] Connection header received
0812 16:36:11.782292 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:11.782299 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:11.782359 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:11.782492 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:11.782741 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.783691 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.784823 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.785330 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.785465 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.786233 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:11.786237 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:11.786238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:11.786428 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:11.786464 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:11.786475 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:11.786553 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:11.786661 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 16:36:11.786755 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260812 16:36:11.787509 7945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e03002d1ba40269cb3940684ea7296" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 35537 } }
I20260812 16:36:11.787724 7945 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:11.787755 7945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e03002d1ba40269cb3940684ea7296, State: Initialized, Role: FOLLOWER
I20260812 16:36:11.787971 7945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e03002d1ba40269cb3940684ea7296" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 35537 } }
I20260812 16:36:11.788053 7945 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:11.788076 7945 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:11.788148 7945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:11.788751 7945 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e03002d1ba40269cb3940684ea7296" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 35537 } }
I20260812 16:36:11.788872 7945 leader_election.cc:302] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87e03002d1ba40269cb3940684ea7296; no voters:
I20260812 16:36:11.789062 7945 leader_election.cc:288] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:11.789119 7950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:11.789327 7950 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296 [term 1 LEADER]: Becoming Leader. State: Replica: 87e03002d1ba40269cb3940684ea7296, State: Running, Role: LEADER
I20260812 16:36:11.789417 7950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e03002d1ba40269cb3940684ea7296" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 35537 } }
I20260812 16:36:11.789480 7945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:11.789865 7953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87e03002d1ba40269cb3940684ea7296. Latest consensus state: current_term: 1 leader_uuid: "87e03002d1ba40269cb3940684ea7296" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e03002d1ba40269cb3940684ea7296" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 35537 } } }
I20260812 16:36:11.789975 7953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:11.789867 7951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87e03002d1ba40269cb3940684ea7296" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e03002d1ba40269cb3940684ea7296" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 35537 } } }
I20260812 16:36:11.790091 7951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:11.790216 7960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:11.790783 7960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:11.792320 7960 catalog_manager.cc:1347] Generated new cluster ID: 49161809aca34cdeb990f88bf16a09d3
I20260812 16:36:11.792373 7960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:11.807011 7960 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:11.807808 7960 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:11.818698 7960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296: Generated new TSK 0
I20260812 16:36:11.818941 7960 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:11.822232 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:35537
--builtin_ntp_servers=127.19.238.84:38725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:36:11.914232 7975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:11.914469 7975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:11.914516 7975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:11.915696 7975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:11.915763 7975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:11.915808 7975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:36:11.917265 7975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:38725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:35537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:11.917549 7975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:11.917856 7975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:11.924218 7981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:11.924225 7983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:11.924351 7975 server_base.cc:1034] running on GCE node
W20260812 16:36:11.924225 7985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:11.924747 7975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:11.925011 7975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:11.926175 7975 hybrid_clock.cc:648] HybridClock initialized: now 1786552571926163 us; error 32 us; skew 500 ppm
I20260812 16:36:11.927475 7975 webserver.cc:460] Webserver started at http://127.19.238.65:44571/ using document root <none> and password file <none>
I20260812 16:36:11.927726 7975 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:11.927800 7975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:11.927961 7975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:11.929069 7975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "cfad3aeaa3294f1687ff43e80e9e21e4"
format_stamp: "Formatted at 2026-08-12 16:36:11 on dist-test-slave-mvvj"
server_key: "a322b561cb313ada38b6ae80f7647334"
server_key_iv: "82d2036bee9837ada2f2f8789d213732"
server_key_version: "encryptionkey@0"
I20260812 16:36:11.929440 7975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "cfad3aeaa3294f1687ff43e80e9e21e4"
format_stamp: "Formatted at 2026-08-12 16:36:11 on dist-test-slave-mvvj"
server_key: "a322b561cb313ada38b6ae80f7647334"
server_key_iv: "82d2036bee9837ada2f2f8789d213732"
server_key_version: "encryptionkey@0"
I20260812 16:36:11.930814 7975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:36:11.931596 7991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:11.931769 7975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 16:36:11.931891 7975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "cfad3aeaa3294f1687ff43e80e9e21e4"
format_stamp: "Formatted at 2026-08-12 16:36:11 on dist-test-slave-mvvj"
server_key: "a322b561cb313ada38b6ae80f7647334"
server_key_iv: "82d2036bee9837ada2f2f8789d213732"
server_key_version: "encryptionkey@0"
I20260812 16:36:11.931977 7975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:11.951574 7975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:11.952029 7975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:11.952196 7975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:11.952450 7975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:36:11.952811 7975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:36:11.952867 7975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:11.952911 7975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:36:11.952946 7975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:11.959113 7975 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:35035
I20260812 16:36:11.959157 8120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:35035 every 8 connection(s)
I20260812 16:36:11.959654 7975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:36:11.965602 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.960545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:60025 (local address 127.19.238.126:35537)
0812 16:36:11.960623 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:11.960626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:11.960868 (+ 242us) server_negotiation.cc:407] Connection header received
0812 16:36:11.961602 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:11.961603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:11.961617 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:11.961677 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:11.962408 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.963027 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.964121 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.964340 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.964430 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.965290 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:11.965291 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:11.965292 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:11.965297 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:11.965314 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:11.965319 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:11.965377 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:11.965473 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 16:36:11.965508 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 16:36:11.965826 8124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:11.960516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:35537 (local address 127.19.238.65:60025)
0812 16:36:11.960717 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:11.960762 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:11.961482 (+ 720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:11.961751 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:11.961755 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:11.961765 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:11.962311 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:11.962319 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.963138 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.963140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:11.964024 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:11.964027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:11.964443 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:11.964444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:11.964587 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:11.965114 (+ 527us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:36:11.965115 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:11.965164 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:36:11.965190 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:11.965381 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:11.965386 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:11.965389 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:36:11.965476 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 16:36:11.965583 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
I20260812 16:36:11.966357 8121 heartbeater.cc:347] Connected to a master server at 127.19.238.126:35537
I20260812 16:36:11.966449 8121 heartbeater.cc:464] Registering TS with master...
I20260812 16:36:11.966647 8121 heartbeater.cc:511] Master 127.19.238.126:35537 requested a full tablet report, sending...
I20260812 16:36:11.967202 7906 ts_manager.cc:194] Registered new tserver with Master: cfad3aeaa3294f1687ff43e80e9e21e4 (127.19.238.65:35035)
I20260812 16:36:11.967916 7906 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.238.65:60025
I20260812 16:36:11.968822 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 7975
I20260812 16:36:11.968940 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:36:11.969156 20409 external_mini_cluster.cc:1428] Setting key 89089f4be11b10f0129c84aadd4e591e
I20260812 16:36:11.970099 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:35537
--builtin_ntp_servers=127.19.238.84:38725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:36:12.055395 8125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:12.055615 8125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:12.055637 8125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:12.056854 8125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:12.056901 8125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:12.056927 8125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:36:12.058339 8125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:38725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:35537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:12.058607 8125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:12.058930 8125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:12.064383 8133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:12.064457 8130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:12.064525 8125 server_base.cc:1034] running on GCE node
W20260812 16:36:12.064383 8131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:12.064885 8125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:12.065171 8125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:12.066362 8125 hybrid_clock.cc:648] HybridClock initialized: now 1786552572066394 us; error 77 us; skew 500 ppm
I20260812 16:36:12.067677 8125 webserver.cc:460] Webserver started at http://127.19.238.66:33259/ using document root <none> and password file <none>
I20260812 16:36:12.068058 8125 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:12.068140 8125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:12.068312 8125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:12.069447 8125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "c05c61d0cb95480ca5c840270e60b563"
format_stamp: "Formatted at 2026-08-12 16:36:12 on dist-test-slave-mvvj"
server_key: "92a04361bb79135382974fe71c6e64ce"
server_key_iv: "1b241fa39b432a05f8cc32dbd2508c78"
server_key_version: "encryptionkey@0"
I20260812 16:36:12.069835 8125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "c05c61d0cb95480ca5c840270e60b563"
format_stamp: "Formatted at 2026-08-12 16:36:12 on dist-test-slave-mvvj"
server_key: "92a04361bb79135382974fe71c6e64ce"
server_key_iv: "1b241fa39b432a05f8cc32dbd2508c78"
server_key_version: "encryptionkey@0"
I20260812 16:36:12.071200 8125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:36:12.072000 8141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:12.072181 8125 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:12.072285 8125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "c05c61d0cb95480ca5c840270e60b563"
format_stamp: "Formatted at 2026-08-12 16:36:12 on dist-test-slave-mvvj"
server_key: "92a04361bb79135382974fe71c6e64ce"
server_key_iv: "1b241fa39b432a05f8cc32dbd2508c78"
server_key_version: "encryptionkey@0"
I20260812 16:36:12.072373 8125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:12.101146 8125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:12.101531 8125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:12.101706 8125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:12.101969 8125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:36:12.102361 8125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:36:12.102421 8125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:12.102458 8125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:36:12.102507 8125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:12.108917 8125 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:45169
I20260812 16:36:12.108971 8269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:45169 every 8 connection(s)
I20260812 16:36:12.109336 8125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:36:12.115000 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:12.110188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:56105 (local address 127.19.238.126:35537)
0812 16:36:12.110282 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:12.110284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:12.110516 (+ 232us) server_negotiation.cc:407] Connection header received
0812 16:36:12.111140 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:12.111141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:12.111155 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:12.111203 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:12.111962 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.112468 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.113485 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.113691 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.113772 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:12.114689 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:12.114691 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:12.114691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:12.114696 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:12.114720 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:12.114724 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:12.114787 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:12.114886 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 16:36:12.114916 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 16:36:12.115262 8272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:12.110135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:35537 (local address 127.19.238.66:56105)
0812 16:36:12.110382 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:12.110419 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:12.111008 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:12.111276 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:12.111281 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:12.111291 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:12.111822 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:12.111874 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.112580 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.112581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:12.113382 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:12.113384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.113787 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.113788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:12.113933 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:12.114506 (+ 573us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:36:12.114506 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:12.114539 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:36:12.114590 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:12.114789 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:12.114796 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:12.114800 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:36:12.114889 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 16:36:12.115001 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":99,"threads_started":1}
I20260812 16:36:12.115872 8270 heartbeater.cc:347] Connected to a master server at 127.19.238.126:35537
I20260812 16:36:12.115979 8270 heartbeater.cc:464] Registering TS with master...
I20260812 16:36:12.116184 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 8125
I20260812 16:36:12.116209 8270 heartbeater.cc:511] Master 127.19.238.126:35537 requested a full tablet report, sending...
I20260812 16:36:12.116322 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:36:12.116568 20409 external_mini_cluster.cc:1428] Setting key b88a694b91533979a8bd65cd36444ee4
I20260812 16:36:12.116653 7906 ts_manager.cc:194] Registered new tserver with Master: c05c61d0cb95480ca5c840270e60b563 (127.19.238.66:45169)
I20260812 16:36:12.117106 7906 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.238.66:56105
I20260812 16:36:12.117702 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:35537
--builtin_ntp_servers=127.19.238.84:38725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:36:12.206372 8274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:12.206614 8274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:12.206636 8274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:12.207960 8274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:12.208014 8274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:12.208040 8274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:36:12.209434 8274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:38725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:35537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:12.209705 8274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:12.210038 8274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:12.215330 8287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:12.215330 8283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:12.215384 8282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:12.215698 8274 server_base.cc:1034] running on GCE node
I20260812 16:36:12.216085 8274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:12.216351 8274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:12.217499 8274 hybrid_clock.cc:648] HybridClock initialized: now 1786552572217462 us; error 55 us; skew 500 ppm
I20260812 16:36:12.218941 8274 webserver.cc:460] Webserver started at http://127.19.238.67:45749/ using document root <none> and password file <none>
I20260812 16:36:12.219241 8274 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:12.219316 8274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:12.219481 8274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:12.220695 8274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "468cfac722a14dedb2150244261619fb"
format_stamp: "Formatted at 2026-08-12 16:36:12 on dist-test-slave-mvvj"
server_key: "126d602bc36474ca901a7fd4da1fae6f"
server_key_iv: "e55a1d248bbc81b42362ae56e7ff470e"
server_key_version: "encryptionkey@0"
I20260812 16:36:12.221094 8274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "468cfac722a14dedb2150244261619fb"
format_stamp: "Formatted at 2026-08-12 16:36:12 on dist-test-slave-mvvj"
server_key: "126d602bc36474ca901a7fd4da1fae6f"
server_key_iv: "e55a1d248bbc81b42362ae56e7ff470e"
server_key_version: "encryptionkey@0"
I20260812 16:36:12.222545 8274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:36:12.223412 8295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:12.223620 8274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:36:12.223716 8274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "468cfac722a14dedb2150244261619fb"
format_stamp: "Formatted at 2026-08-12 16:36:12 on dist-test-slave-mvvj"
server_key: "126d602bc36474ca901a7fd4da1fae6f"
server_key_iv: "e55a1d248bbc81b42362ae56e7ff470e"
server_key_version: "encryptionkey@0"
I20260812 16:36:12.223803 8274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:12.243808 8274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:12.244243 8274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:12.244405 8274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:12.244680 8274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:36:12.245038 8274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:36:12.245101 8274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:12.245149 8274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:36:12.245174 8274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:12.251209 8274 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:39431
I20260812 16:36:12.251292 8421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:39431 every 8 connection(s)
I20260812 16:36:12.251649 8274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:36:12.253777 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 8274
I20260812 16:36:12.253901 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:36:12.254143 20409 external_mini_cluster.cc:1428] Setting key 38474a01e94e5ee0ba3055fef0358445
I20260812 16:36:12.257778 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:12.252522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:47995 (local address 127.19.238.126:35537)
0812 16:36:12.252594 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:12.252596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:12.252850 (+ 254us) server_negotiation.cc:407] Connection header received
0812 16:36:12.253417 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:12.253419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:12.253433 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:12.253484 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:12.254483 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.255211 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.256377 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.256593 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.256682 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:12.257472 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:12.257474 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:12.257474 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:12.257479 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:12.257496 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:12.257500 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:12.257564 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:12.257667 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 16:36:12.257695 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 16:36:12.258008 8426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:12.252478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:35537 (local address 127.19.238.67:47995)
0812 16:36:12.252718 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:12.252754 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:12.253298 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:12.253579 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:12.253584 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:12.253594 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:12.254371 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:12.254377 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.255303 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.255304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:12.256278 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:12.256281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.256694 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.256695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:12.256834 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:12.257326 (+ 492us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:36:12.257327 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:12.257357 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:36:12.257381 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:12.257568 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:12.257573 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:12.257576 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:36:12.257657 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 16:36:12.257767 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":111,"threads_started":1}
I20260812 16:36:12.258599 8423 heartbeater.cc:347] Connected to a master server at 127.19.238.126:35537
I20260812 16:36:12.258687 8423 heartbeater.cc:464] Registering TS with master...
I20260812 16:36:12.258834 8423 heartbeater.cc:511] Master 127.19.238.126:35537 requested a full tablet report, sending...
I20260812 16:36:12.259187 7906 ts_manager.cc:194] Registered new tserver with Master: 468cfac722a14dedb2150244261619fb (127.19.238.67:39431)
I20260812 16:36:12.259518 7906 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.238.67:47995
I20260812 16:36:12.266153 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:36:12.270705 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:12.267661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36004 (local address 127.19.238.126:35537)
0812 16:36:12.267747 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:12.267749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:12.267881 (+ 132us) server_negotiation.cc:407] Connection header received
0812 16:36:12.267988 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:12.267989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:12.268000 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:12.268048 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:12.268248 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.268740 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.269489 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.269703 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.269790 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:12.270453 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:12.270454 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:12.270454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:12.270459 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:12.270473 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:12.270477 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:12.270527 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:12.270593 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 16:36:12.270616 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 16:36:12.272321 7906 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36004:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:36:12.274109 7906 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:36:12.281138 8434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:12.276704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:39431 (local address 127.0.0.1:40888)
0812 16:36:12.277270 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:12.277278 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:12.277321 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:12.277584 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:12.277585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:12.277586 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:12.277632 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:12.277634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.278488 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.278489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:12.279890 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:12.279893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.280832 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.280833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:12.280891 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:12.280902 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 16:36:12.280995 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 16:36:12.281057 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":31,"thread_start_us":67,"threads_started":1}
I20260812 16:36:12.281154 8440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:12.276903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40888 (local address 127.19.238.67:39431)
0812 16:36:12.277121 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:12.277125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:12.277380 (+ 255us) server_negotiation.cc:407] Connection header received
0812 16:36:12.277426 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:12.277428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:12.277504 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:12.277586 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:12.277688 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.278391 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.279994 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.280746 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.280823 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:12.280912 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:12.281028 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 16:36:12.281069 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":108,"threads_started":1}
I20260812 16:36:12.282060 8439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:12.276529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:35035 (local address 127.0.0.1:48496)
0812 16:36:12.277109 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:12.277120 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:12.277184 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:12.277385 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:12.277385 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:12.277387 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:12.277423 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:12.277425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.278703 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.278704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:12.279890 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:12.279893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.281834 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.281835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:12.281895 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:12.281899 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:36:12.281956 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 16:36:12.281994 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":2,"thread_start_us":54,"threads_started":1}
I20260812 16:36:12.282104 8438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:12.276634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48496 (local address 127.19.238.65:35035)
0812 16:36:12.276832 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:12.276837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:12.277182 (+ 345us) server_negotiation.cc:407] Connection header received
0812 16:36:12.277259 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:12.277261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:12.277278 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:12.277495 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:12.277578 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.278638 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.280954 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.281713 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.281821 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:12.281901 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:12.281966 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 16:36:12.282005 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":111,"threads_started":1}
I20260812 16:36:12.282198 8433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:12.276485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:45169 (local address 127.0.0.1:56880)
0812 16:36:12.276671 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:12.276711 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:12.276782 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:12.277120 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:12.277123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:12.277126 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:12.277199 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:12.277201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.278588 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.278588 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:12.280939 (+ 2351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:12.280943 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.281934 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.281935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:12.281999 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:12.282002 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:36:12.282067 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 16:36:12.282111 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":1259,"thread_start_us":71,"threads_started":1}
I20260812 16:36:12.282393 8436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:12.276567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56880 (local address 127.19.238.66:45169)
0812 16:36:12.276890 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:12.276895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:12.276906 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:36:12.276986 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:12.276988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:12.277006 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:12.277102 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:12.277301 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.278516 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.281032 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.281748 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.281812 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:12.281897 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:12.282264 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 16:36:12.282302 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":110,"threads_started":1}
I20260812 16:36:12.282286 8347 tablet_service.cc:1467] Processing CreateTablet for tablet a93c5c855ea54810aea52b6f8a77eff2 (DEFAULT_TABLE table=test-table [id=b3924dd249914768a86f96057ed51323]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:36:12.282676 8347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a93c5c855ea54810aea52b6f8a77eff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:12.283119 8048 tablet_service.cc:1467] Processing CreateTablet for tablet a93c5c855ea54810aea52b6f8a77eff2 (DEFAULT_TABLE table=test-table [id=b3924dd249914768a86f96057ed51323]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:36:12.283486 8048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a93c5c855ea54810aea52b6f8a77eff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:12.283581 8196 tablet_service.cc:1467] Processing CreateTablet for tablet a93c5c855ea54810aea52b6f8a77eff2 (DEFAULT_TABLE table=test-table [id=b3924dd249914768a86f96057ed51323]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:36:12.283962 8196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a93c5c855ea54810aea52b6f8a77eff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:12.285948 8441 tablet_bootstrap.cc:492] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb: Bootstrap starting.
I20260812 16:36:12.286602 8441 tablet_bootstrap.cc:654] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:12.286984 8441 log.cc:824] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:12.287151 8442 tablet_bootstrap.cc:492] T a93c5c855ea54810aea52b6f8a77eff2 P cfad3aeaa3294f1687ff43e80e9e21e4: Bootstrap starting.
I20260812 16:36:12.287262 8444 tablet_bootstrap.cc:492] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563: Bootstrap starting.
I20260812 16:36:12.287905 8442 tablet_bootstrap.cc:654] T a93c5c855ea54810aea52b6f8a77eff2 P cfad3aeaa3294f1687ff43e80e9e21e4: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:12.288002 8444 tablet_bootstrap.cc:654] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:12.288256 8442 log.cc:824] T a93c5c855ea54810aea52b6f8a77eff2 P cfad3aeaa3294f1687ff43e80e9e21e4: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:12.288290 8441 tablet_bootstrap.cc:492] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb: No bootstrap required, opened a new log
I20260812 16:36:12.288350 8441 ts_tablet_manager.cc:1397] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:36:12.288407 8444 log.cc:824] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:12.289002 8442 tablet_bootstrap.cc:492] T a93c5c855ea54810aea52b6f8a77eff2 P cfad3aeaa3294f1687ff43e80e9e21e4: No bootstrap required, opened a new log
I20260812 16:36:12.289088 8442 ts_tablet_manager.cc:1397] T a93c5c855ea54810aea52b6f8a77eff2 P cfad3aeaa3294f1687ff43e80e9e21e4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 16:36:12.289309 8444 tablet_bootstrap.cc:492] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563: No bootstrap required, opened a new log
I20260812 16:36:12.289389 8444 ts_tablet_manager.cc:1397] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 16:36:12.291007 8441 raft_consensus.cc:348] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05c61d0cb95480ca5c840270e60b563" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45169 } } peers { permanent_uuid: "cfad3aeaa3294f1687ff43e80e9e21e4" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35035 } } peers { permanent_uuid: "468cfac722a14dedb2150244261619fb" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39431 } }
I20260812 16:36:12.291211 8441 raft_consensus.cc:374] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:12.291275 8441 raft_consensus.cc:729] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468cfac722a14dedb2150244261619fb, State: Initialized, Role: FOLLOWER
I20260812 16:36:12.291412 8441 consensus_queue.cc:260] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05c61d0cb95480ca5c840270e60b563" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45169 } } peers { permanent_uuid: "cfad3aeaa3294f1687ff43e80e9e21e4" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35035 } } peers { permanent_uuid: "468cfac722a14dedb2150244261619fb" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39431 } }
I20260812 16:36:12.291710 8441 ts_tablet_manager.cc:1428] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:36:12.291738 8423 heartbeater.cc:503] Master 127.19.238.126:35537 was elected leader, sending a full tablet report...
I20260812 16:36:12.292068 8442 raft_consensus.cc:348] T a93c5c855ea54810aea52b6f8a77eff2 P cfad3aeaa3294f1687ff43e80e9e21e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05c61d0cb95480ca5c840270e60b563" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45169 } } peers { permanent_uuid: "cfad3aeaa3294f1687ff43e80e9e21e4" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35035 } } peers { permanent_uuid: "468cfac722a14dedb2150244261619fb" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39431 } }
I20260812 16:36:12.292309 8442 raft_consensus.cc:374] T a93c5c855ea54810aea52b6f8a77eff2 P cfad3aeaa3294f1687ff43e80e9e21e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:12.292375 8442 raft_consensus.cc:729] T a93c5c855ea54810aea52b6f8a77eff2 P cfad3aeaa3294f1687ff43e80e9e21e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfad3aeaa3294f1687ff43e80e9e21e4, State: Initialized, Role: FOLLOWER
I20260812 16:36:12.292331 8444 raft_consensus.cc:348] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05c61d0cb95480ca5c840270e60b563" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45169 } } peers { permanent_uuid: "cfad3aeaa3294f1687ff43e80e9e21e4" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35035 } } peers { permanent_uuid: "468cfac722a14dedb2150244261619fb" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39431 } }
I20260812 16:36:12.292503 8444 raft_consensus.cc:374] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:12.292570 8444 raft_consensus.cc:729] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c05c61d0cb95480ca5c840270e60b563, State: Initialized, Role: FOLLOWER
I20260812 16:36:12.292567 8442 consensus_queue.cc:260] T a93c5c855ea54810aea52b6f8a77eff2 P cfad3aeaa3294f1687ff43e80e9e21e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05c61d0cb95480ca5c840270e60b563" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45169 } } peers { permanent_uuid: "cfad3aeaa3294f1687ff43e80e9e21e4" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35035 } } peers { permanent_uuid: "468cfac722a14dedb2150244261619fb" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39431 } }
I20260812 16:36:12.292757 8444 consensus_queue.cc:260] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05c61d0cb95480ca5c840270e60b563" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45169 } } peers { permanent_uuid: "cfad3aeaa3294f1687ff43e80e9e21e4" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35035 } } peers { permanent_uuid: "468cfac722a14dedb2150244261619fb" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39431 } }
I20260812 16:36:12.292853 8442 ts_tablet_manager.cc:1428] T a93c5c855ea54810aea52b6f8a77eff2 P cfad3aeaa3294f1687ff43e80e9e21e4: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260812 16:36:12.292989 8121 heartbeater.cc:503] Master 127.19.238.126:35537 was elected leader, sending a full tablet report...
I20260812 16:36:12.293011 8444 ts_tablet_manager.cc:1428] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260812 16:36:12.293118 8270 heartbeater.cc:503] Master 127.19.238.126:35537 was elected leader, sending a full tablet report...
W20260812 16:36:12.360041 8271 tablet.cc:2369] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:36:12.460583 8123 tablet.cc:2369] T a93c5c855ea54810aea52b6f8a77eff2 P cfad3aeaa3294f1687ff43e80e9e21e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:36:12.502322 8424 tablet.cc:2369] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:36:12.508478 8451 raft_consensus.cc:482] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:36:12.508606 8451 raft_consensus.cc:504] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05c61d0cb95480ca5c840270e60b563" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45169 } } peers { permanent_uuid: "cfad3aeaa3294f1687ff43e80e9e21e4" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35035 } } peers { permanent_uuid: "468cfac722a14dedb2150244261619fb" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39431 } }
I20260812 16:36:12.508955 8451 leader_election.cc:288] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfad3aeaa3294f1687ff43e80e9e21e4 (127.19.238.65:35035), 468cfac722a14dedb2150244261619fb (127.19.238.67:39431)
I20260812 16:36:12.512054 8438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:12.509239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:53745 (local address 127.19.238.65:35035)
0812 16:36:12.509328 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:12.509330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:12.509341 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:36:12.509375 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:12.509377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:12.509389 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:12.509528 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:12.509762 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.510303 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.511302 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.511667 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.511722 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:12.511728 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:12.511935 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 16:36:12.511979 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 16:36:12.512056 8272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:12.509134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:35035 (local address 127.19.238.66:53745)
0812 16:36:12.509200 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:12.509213 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:12.509262 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:12.509585 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:12.509587 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:12.509589 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:12.509671 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:12.509674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.510394 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.510396 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:12.511202 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:12.511204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.511754 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.511756 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:12.511853 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:12.511857 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:36:12.511915 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 16:36:12.511973 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 16:36:12.512112 8440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:12.509360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:44425 (local address 127.19.238.67:39431)
0812 16:36:12.509439 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:12.509441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:12.509555 (+ 114us) server_negotiation.cc:407] Connection header received
0812 16:36:12.509609 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:12.509610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:12.509619 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:12.509683 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:12.509871 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.510332 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.511339 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.511690 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.511782 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:12.511790 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:12.511985 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 16:36:12.512029 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 16:36:12.512158 8452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:12.509270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:39431 (local address 127.19.238.66:44425)
0812 16:36:12.509478 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:12.509490 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:12.509549 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:12.509695 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:12.509696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:12.509697 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:12.509797 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:12.509798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.510446 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.510448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:12.511213 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:12.511215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.511808 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.511809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:12.511878 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:12.511882 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:36:12.511969 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 16:36:12.512006 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":142,"threads_started":1}
I20260812 16:36:12.512413 8069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a93c5c855ea54810aea52b6f8a77eff2" candidate_uuid: "c05c61d0cb95480ca5c840270e60b563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfad3aeaa3294f1687ff43e80e9e21e4" is_pre_election: true
I20260812 16:36:12.512586 8069 raft_consensus.cc:2454] T a93c5c855ea54810aea52b6f8a77eff2 P cfad3aeaa3294f1687ff43e80e9e21e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c05c61d0cb95480ca5c840270e60b563 in term 0.
I20260812 16:36:12.512573 8369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a93c5c855ea54810aea52b6f8a77eff2" candidate_uuid: "c05c61d0cb95480ca5c840270e60b563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468cfac722a14dedb2150244261619fb" is_pre_election: true
I20260812 16:36:12.512743 8369 raft_consensus.cc:2454] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c05c61d0cb95480ca5c840270e60b563 in term 0.
I20260812 16:36:12.512838 8145 leader_election.cc:302] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c05c61d0cb95480ca5c840270e60b563, cfad3aeaa3294f1687ff43e80e9e21e4; no voters:
I20260812 16:36:12.512984 8451 raft_consensus.cc:2781] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:36:12.513044 8451 raft_consensus.cc:482] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:36:12.513070 8451 raft_consensus.cc:3037] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:12.513778 8451 raft_consensus.cc:504] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05c61d0cb95480ca5c840270e60b563" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45169 } } peers { permanent_uuid: "cfad3aeaa3294f1687ff43e80e9e21e4" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35035 } } peers { permanent_uuid: "468cfac722a14dedb2150244261619fb" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39431 } }
I20260812 16:36:12.513912 8451 leader_election.cc:288] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 [CANDIDATE]: Term 1 election: Requested vote from peers cfad3aeaa3294f1687ff43e80e9e21e4 (127.19.238.65:35035), 468cfac722a14dedb2150244261619fb (127.19.238.67:39431)
I20260812 16:36:12.514111 8369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a93c5c855ea54810aea52b6f8a77eff2" candidate_uuid: "c05c61d0cb95480ca5c840270e60b563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468cfac722a14dedb2150244261619fb"
I20260812 16:36:12.514135 8069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a93c5c855ea54810aea52b6f8a77eff2" candidate_uuid: "c05c61d0cb95480ca5c840270e60b563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfad3aeaa3294f1687ff43e80e9e21e4"
I20260812 16:36:12.514201 8369 raft_consensus.cc:3037] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:12.514223 8069 raft_consensus.cc:3037] T a93c5c855ea54810aea52b6f8a77eff2 P cfad3aeaa3294f1687ff43e80e9e21e4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:12.515002 8369 raft_consensus.cc:2454] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c05c61d0cb95480ca5c840270e60b563 in term 1.
I20260812 16:36:12.515014 8069 raft_consensus.cc:2454] T a93c5c855ea54810aea52b6f8a77eff2 P cfad3aeaa3294f1687ff43e80e9e21e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c05c61d0cb95480ca5c840270e60b563 in term 1.
I20260812 16:36:12.515180 8145 leader_election.cc:302] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468cfac722a14dedb2150244261619fb, c05c61d0cb95480ca5c840270e60b563; no voters:
I20260812 16:36:12.515337 8451 raft_consensus.cc:2781] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:12.515497 8451 raft_consensus.cc:686] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 [term 1 LEADER]: Becoming Leader. State: Replica: c05c61d0cb95480ca5c840270e60b563, State: Running, Role: LEADER
I20260812 16:36:12.515745 8451 consensus_queue.cc:237] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05c61d0cb95480ca5c840270e60b563" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45169 } } peers { permanent_uuid: "cfad3aeaa3294f1687ff43e80e9e21e4" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35035 } } peers { permanent_uuid: "468cfac722a14dedb2150244261619fb" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39431 } }
I20260812 16:36:12.516741 7905 catalog_manager.cc:5495] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 reported cstate change: term changed from 0 to 1, leader changed from <none> to c05c61d0cb95480ca5c840270e60b563 (127.19.238.66). New cstate: current_term: 1 leader_uuid: "c05c61d0cb95480ca5c840270e60b563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05c61d0cb95480ca5c840270e60b563" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 45169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cfad3aeaa3294f1687ff43e80e9e21e4" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468cfac722a14dedb2150244261619fb" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39431 } health_report { overall_health: UNKNOWN } } }
I20260812 16:36:12.563889 8436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:12.561410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56896 (local address 127.19.238.66:45169)
0812 16:36:12.561481 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:12.561482 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:12.561489 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:36:12.561531 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:12.561532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:12.561554 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:12.561638 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:36:12.561780 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.562262 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.563014 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:12.563218 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:12.563296 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:12.563460 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:36:12.563573 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:36:12.563676 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:12.563695 (+ 19us) server_negotiation.cc:299] Negotiation successful
0812 16:36:12.563716 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 16:36:12.565941 8069 raft_consensus.cc:1261] T a93c5c855ea54810aea52b6f8a77eff2 P cfad3aeaa3294f1687ff43e80e9e21e4 [term 1 FOLLOWER]: Refusing update from remote peer c05c61d0cb95480ca5c840270e60b563: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:36:12.565958 8369 raft_consensus.cc:1261] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb [term 1 FOLLOWER]: Refusing update from remote peer c05c61d0cb95480ca5c840270e60b563: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:36:12.566320 8454 consensus_queue.cc:1035] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfad3aeaa3294f1687ff43e80e9e21e4" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 35035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:36:12.566458 8451 consensus_queue.cc:1035] T a93c5c855ea54810aea52b6f8a77eff2 P c05c61d0cb95480ca5c840270e60b563 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468cfac722a14dedb2150244261619fb" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 39431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:36:12.569404 8467 mvcc.cc:205] Tried to move back new op lower bound from 7317719337227251712 to 7317719337025486848. Current Snapshot: MvccSnapshot[applied={T|T < 7317719337227251712}]
I20260812 16:36:12.571812 7906 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:36:12.571945 7906 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:36:12.572773 7906 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87e03002d1ba40269cb3940684ea7296: Sending DeleteTablet for 3 replicas of tablet a93c5c855ea54810aea52b6f8a77eff2
I20260812 16:36:12.573081 8196 tablet_service.cc:1514] Processing DeleteTablet for tablet a93c5c855ea54810aea52b6f8a77eff2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:36:12 UTC) from {username='slave'} at 127.0.0.1:56880
I20260812 16:36:12.573139 8048 tablet_service.cc:1514] Processing DeleteTablet for tablet a93c5c855ea54810aea52b6f8a77eff2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:36:12 UTC) from {username='slave'} at 127.0.0.1:48496
I20260812 16:36:12.573203 8347 tablet_service.cc:1514] Processing DeleteTablet for tablet a93c5c855ea54810aea52b6f8a77eff2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:36:12 UTC) from {username='slave'} at 127.0.0.1:40888
I20260812 16:36:12.573455 8477 tablet_replica.cc:330] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb: stopping tablet replica
I20260812 16:36:12.573576 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 7975
I20260812 16:36:12.573598 8476 tablet_replica.cc:330] T a93c5c855ea54810aea52b6f8a77eff2 P cfad3aeaa3294f1687ff43e80e9e21e4: stopping tablet replica
I20260812 16:36:12.573606 8477 raft_consensus.cc:2229] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:36:12.574188 8477 raft_consensus.cc:2258] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:36:12.578480 8477 ts_tablet_manager.cc:1905] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:36:12.580650 8477 ts_tablet_manager.cc:1918] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:36:12.580739 8477 log.cc:1196] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/a93c5c855ea54810aea52b6f8a77eff2
I20260812 16:36:12.581058 8477 ts_tablet_manager.cc:1939] T a93c5c855ea54810aea52b6f8a77eff2 P 468cfac722a14dedb2150244261619fb: Deleting consensus metadata
I20260812 16:36:12.581617 7889 catalog_manager.cc:4841] TS 468cfac722a14dedb2150244261619fb (127.19.238.67:39431): tablet a93c5c855ea54810aea52b6f8a77eff2 (table test-table [id=b3924dd249914768a86f96057ed51323]) successfully deleted
W20260812 16:36:12.582712 7889 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.238.65:35035 (error 108)
I20260812 16:36:12.583035 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 8125
I20260812 16:36:12.583245 8434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:12.583109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:35035 (local address 127.0.0.1:48500)
0812 16:36:12.583159 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:12.583207 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:35035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 16:36:12.584684 7889 catalog_manager.cc:4584] TS cfad3aeaa3294f1687ff43e80e9e21e4 (127.19.238.65:35035): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a93c5c855ea54810aea52b6f8a77eff2: Network error: Client connection negotiation failed: client connection to 127.19.238.65:35035: connect: Connection refused (error 111)
I20260812 16:36:12.591440 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 8274
I20260812 16:36:12.591481 8434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:12.591322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:45169 (local address 127.0.0.1:56902)
0812 16:36:12.591403 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:12.591447 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.66:45169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 16:36:12.599076 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 7870
2026-08-12T16:36:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (983 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:12 dist-test-slave-mvvj krb5kdc[8481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:12 dist-test-slave-mvvj krb5kdc[8481](info): set up 2 sockets
Aug 12 16:36:12 dist-test-slave-mvvj krb5kdc[8481](info): commencing operation
krb5kdc: starting...
W20260812 16:36:13.711371 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:13 dist-test-slave-mvvj krb5kdc[8481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:36:13.843354 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:33623
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:44541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:33623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:13.928915 8495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:13.929142 8495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:13.929165 8495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:13.930267 8495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:13.930305 8495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:13.930320 8495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:13.930332 8495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:13.930349 8495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:13.931756 8495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:44541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:33623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:33623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:13.932044 8495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:13.932293 8495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:13.937145 8502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:13.937342 8495 server_base.cc:1034] running on GCE node
W20260812 16:36:13.937139 8504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:13.937139 8501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:13.937788 8495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:13.938054 8495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:13.939227 8495 hybrid_clock.cc:648] HybridClock initialized: now 1786552573939196 us; error 46 us; skew 500 ppm
Aug 12 16:36:13 dist-test-slave-mvvj krb5kdc[8481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552573, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:13.945513 8495 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:36:13.946089 8495 webserver.cc:460] Webserver started at http://127.19.238.126:33997/ using document root <none> and password file <none>
I20260812 16:36:13.946349 8495 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:13.946427 8495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:13.946597 8495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:13.947829 8495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "3a437a4e5ba247349dd5d4c51265f74e"
format_stamp: "Formatted at 2026-08-12 16:36:13 on dist-test-slave-mvvj"
server_key: "f8ca0fd33adf84957b8f4f6bd5891400"
server_key_iv: "8bd76ec2d646e24e3b23ad633db024d9"
server_key_version: "encryptionkey@0"
I20260812 16:36:13.948314 8495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "3a437a4e5ba247349dd5d4c51265f74e"
format_stamp: "Formatted at 2026-08-12 16:36:13 on dist-test-slave-mvvj"
server_key: "f8ca0fd33adf84957b8f4f6bd5891400"
server_key_iv: "8bd76ec2d646e24e3b23ad633db024d9"
server_key_version: "encryptionkey@0"
I20260812 16:36:13.949821 8495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:36:13.950634 8512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:13.950814 8495 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:36:13.950933 8495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "3a437a4e5ba247349dd5d4c51265f74e"
format_stamp: "Formatted at 2026-08-12 16:36:13 on dist-test-slave-mvvj"
server_key: "f8ca0fd33adf84957b8f4f6bd5891400"
server_key_iv: "8bd76ec2d646e24e3b23ad633db024d9"
server_key_version: "encryptionkey@0"
I20260812 16:36:13.951030 8495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:13.964601 8495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:13.966754 8495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:13.966987 8495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:13.972254 8495 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:33623
I20260812 16:36:13.972306 8571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:33623 every 8 connection(s)
I20260812 16:36:13.972687 8495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:13.973630 8572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:13.976053 8572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e: Bootstrap starting.
I20260812 16:36:13.976648 8572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:13.976954 8572 log.cc:824] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:13.977723 8572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e: No bootstrap required, opened a new log
I20260812 16:36:13.979091 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 8495
I20260812 16:36:13.979209 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:13.979413 20409 external_mini_cluster.cc:1428] Setting key d2e025f910f5aebf51a56541ffa33e2a
I20260812 16:36:13.979979 8572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a437a4e5ba247349dd5d4c51265f74e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 33623 } }
I20260812 16:36:13.980271 8572 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:13.980358 8572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a437a4e5ba247349dd5d4c51265f74e, State: Initialized, Role: FOLLOWER
I20260812 16:36:13.980504 8572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a437a4e5ba247349dd5d4c51265f74e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 33623 } }
I20260812 16:36:13.980610 8572 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:13.980656 8572 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:13.980707 8572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:13.981690 8572 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a437a4e5ba247349dd5d4c51265f74e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 33623 } }
I20260812 16:36:13.981804 8572 leader_election.cc:302] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a437a4e5ba247349dd5d4c51265f74e; no voters:
I20260812 16:36:13.981962 8572 leader_election.cc:288] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:13.982065 8577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:13.982220 8572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:13.982275 8577 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e [term 1 LEADER]: Becoming Leader. State: Replica: 3a437a4e5ba247349dd5d4c51265f74e, State: Running, Role: LEADER
I20260812 16:36:13.982424 8577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a437a4e5ba247349dd5d4c51265f74e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 33623 } }
I20260812 16:36:13.982877 8578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a437a4e5ba247349dd5d4c51265f74e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a437a4e5ba247349dd5d4c51265f74e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 33623 } } }
I20260812 16:36:13.983017 8578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:13.983217 8580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a437a4e5ba247349dd5d4c51265f74e. Latest consensus state: current_term: 1 leader_uuid: "3a437a4e5ba247349dd5d4c51265f74e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a437a4e5ba247349dd5d4c51265f74e" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 33623 } } }
I20260812 16:36:13.983278 8580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e [sys.catalog]: This master's current role is: LEADER
W20260812 16:36:13.984107 8599 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:36:13.984179 8599 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:36:13.984273 8590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:13.984480 8590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:13.986213 8590 catalog_manager.cc:1347] Generated new cluster ID: efa4d9438f114424b74e4bcbbe6f73f1
I20260812 16:36:13.986335 8590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:36:13 dist-test-slave-mvvj krb5kdc[8481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:36:13.992053 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:13.980080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47332 (local address 127.19.238.126:33623)
0812 16:36:13.980285 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:13.980294 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:13.980321 (+ 27us) server_negotiation.cc:407] Connection header received
0812 16:36:13.980463 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:13.980470 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:13.980553 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:13.980717 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:13.981528 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:13.983023 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:13.984251 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:13.984850 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:13.984961 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:13.988140 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:13.988145 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:13.988145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:13.988158 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:13.990260 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:13.990704 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:13.990707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:13.990707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:13.990839 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:13.991074 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:13.991074 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:13.991075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:13.991401 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:13.991588 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:13.991665 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 16:36:13.991811 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":112,"threads_started":1}
I20260812 16:36:14.007237 8590 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:14.007928 8590 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:14.014750 8590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e: Generated new TSK 0
I20260812 16:36:14.014945 8590 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
I20260812 16:36:14.051784 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:33623
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:44541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:14.136863 8604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:14.137110 8604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:14.137137 8604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:14.138327 8604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:14.138372 8604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:14.138401 8604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:36:14.139926 8604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:44541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:33623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:14.140184 8604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:14.140419 8604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:14.145727 8612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:14.145738 8614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:14.145764 8604 server_base.cc:1034] running on GCE node
W20260812 16:36:14.145743 8611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:14.146191 8604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:14.146474 8604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:14.147648 8604 hybrid_clock.cc:648] HybridClock initialized: now 1786552574147643 us; error 38 us; skew 500 ppm
Aug 12 16:36:14 dist-test-slave-mvvj krb5kdc[8481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552574, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:14.150202 8604 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:36:14.150695 8604 webserver.cc:460] Webserver started at http://127.19.238.65:39011/ using document root <none> and password file <none>
I20260812 16:36:14.150950 8604 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:14.151027 8604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:14.151182 8604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:14.152349 8604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "ce8b41c300184958a65f16d118699094"
format_stamp: "Formatted at 2026-08-12 16:36:14 on dist-test-slave-mvvj"
server_key: "fe429da17d53128f1a59006048b78a3b"
server_key_iv: "071a8355ffac313b5797c914b931df2e"
server_key_version: "encryptionkey@0"
I20260812 16:36:14.152719 8604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "ce8b41c300184958a65f16d118699094"
format_stamp: "Formatted at 2026-08-12 16:36:14 on dist-test-slave-mvvj"
server_key: "fe429da17d53128f1a59006048b78a3b"
server_key_iv: "071a8355ffac313b5797c914b931df2e"
server_key_version: "encryptionkey@0"
I20260812 16:36:14.154095 8604 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:14.154836 8622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:14.155022 8604 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:36:14.155117 8604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "ce8b41c300184958a65f16d118699094"
format_stamp: "Formatted at 2026-08-12 16:36:14 on dist-test-slave-mvvj"
server_key: "fe429da17d53128f1a59006048b78a3b"
server_key_iv: "071a8355ffac313b5797c914b931df2e"
server_key_version: "encryptionkey@0"
I20260812 16:36:14.155215 8604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:14.168550 8604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:14.170468 8604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:14.170636 8604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:14.170884 8604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:36:14.171211 8604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:36:14.171267 8604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:14.171311 8604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:36:14.171337 8604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:14.177637 8604 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:44331
I20260812 16:36:14.177696 8754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:44331 every 8 connection(s)
I20260812 16:36:14.178045 8604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
Aug 12 16:36:14 dist-test-slave-mvvj krb5kdc[8481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552574, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:36:14.187225 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:14.178866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:59485 (local address 127.19.238.126:33623)
0812 16:36:14.178932 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:14.178934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:14.179132 (+ 198us) server_negotiation.cc:407] Connection header received
0812 16:36:14.179659 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:14.179660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:14.179674 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:14.179736 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:14.180753 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.181270 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.182235 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.182433 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.182499 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:14.184770 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:14.184772 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:14.184772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:14.184779 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:14.186033 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:14.186400 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:14.186400 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:14.186401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:14.186452 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:14.186670 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:14.186670 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:14.186670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:14.186775 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:14.186827 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:14.187079 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 16:36:14.187156 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:36:14.187366 8757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:14.178822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:33623 (local address 127.19.238.65:59485)
0812 16:36:14.178991 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:14.179034 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:14.179567 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:14.179790 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:14.179793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:14.180440 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:14.180650 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:14.180656 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.181353 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.181354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:14.182165 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:14.182167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.182531 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.182532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:14.182635 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:14.183175 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:14.183184 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:14.184684 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:14.186111 (+ 1427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:14.186117 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:14.186119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:14.186333 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:14.186513 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:14.186513 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:14.186513 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:14.186614 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:14.186836 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:14.186841 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:14.186966 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 16:36:14.187048 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 16:36:14.187167 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260812 16:36:14.187808 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 8604
I20260812 16:36:14.187938 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:36:14.188131 20409 external_mini_cluster.cc:1428] Setting key d468b78b577938a530732a4a629da011
I20260812 16:36:14.188306 8755 heartbeater.cc:347] Connected to a master server at 127.19.238.126:33623
I20260812 16:36:14.188479 8755 heartbeater.cc:464] Registering TS with master...
I20260812 16:36:14.188681 8755 heartbeater.cc:511] Master 127.19.238.126:33623 requested a full tablet report, sending...
I20260812 16:36:14.189179 8522 ts_manager.cc:194] Registered new tserver with Master: ce8b41c300184958a65f16d118699094 (127.19.238.65:44331)
I20260812 16:36:14.189781 8522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:59485
WARNING: no policy specified for kudu/127.19.238.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
I20260812 16:36:14.219194 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:33623
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:44541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:14.308846 8760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:14.309094 8760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:14.309120 8760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:14.310266 8760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:14.310304 8760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:14.310331 8760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:36:14.311805 8760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:44541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:33623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:14.312228 8760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:14.312482 8760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:14.317741 8765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:14.317750 8766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:14.317759 8768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:14.318316 8760 server_base.cc:1034] running on GCE node
I20260812 16:36:14.318575 8760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:14.318825 8760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:14.319988 8760 hybrid_clock.cc:648] HybridClock initialized: now 1786552574319968 us; error 31 us; skew 500 ppm
Aug 12 16:36:14 dist-test-slave-mvvj krb5kdc[8481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552574, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:14.322582 8760 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:36:14.323093 8760 webserver.cc:460] Webserver started at http://127.19.238.66:33939/ using document root <none> and password file <none>
I20260812 16:36:14.323339 8760 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:14.323411 8760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:14.323568 8760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:14.324723 8760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "786834d7849e4759a008dd872d121f7d"
format_stamp: "Formatted at 2026-08-12 16:36:14 on dist-test-slave-mvvj"
server_key: "043e5ddd06f560501256e7e1bef6758e"
server_key_iv: "23987b4b6c1b757adcd1f04959af0d78"
server_key_version: "encryptionkey@0"
I20260812 16:36:14.325107 8760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "786834d7849e4759a008dd872d121f7d"
format_stamp: "Formatted at 2026-08-12 16:36:14 on dist-test-slave-mvvj"
server_key: "043e5ddd06f560501256e7e1bef6758e"
server_key_iv: "23987b4b6c1b757adcd1f04959af0d78"
server_key_version: "encryptionkey@0"
I20260812 16:36:14.326481 8760 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:36:14.327219 8776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:14.327407 8760 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 16:36:14.327517 8760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "786834d7849e4759a008dd872d121f7d"
format_stamp: "Formatted at 2026-08-12 16:36:14 on dist-test-slave-mvvj"
server_key: "043e5ddd06f560501256e7e1bef6758e"
server_key_iv: "23987b4b6c1b757adcd1f04959af0d78"
server_key_version: "encryptionkey@0"
I20260812 16:36:14.327606 8760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:14.344012 8760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:14.346072 8760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:14.346258 8760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:14.346580 8760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:36:14.346927 8760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:36:14.346992 8760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:14.347028 8760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:36:14.347076 8760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:14.353134 8760 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:41681
I20260812 16:36:14.353188 8913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:41681 every 8 connection(s)
I20260812 16:36:14.353564 8760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:36:14.355580 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 8760
I20260812 16:36:14.355693 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:36:14.355976 20409 external_mini_cluster.cc:1428] Setting key 2e1477f72cdf4a7a387ccdcb94dc5fa4
Aug 12 16:36:14 dist-test-slave-mvvj krb5kdc[8481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552574, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:36:14.363921 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:14.354260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:33389 (local address 127.19.238.126:33623)
0812 16:36:14.354307 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:14.354309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:14.354539 (+ 230us) server_negotiation.cc:407] Connection header received
0812 16:36:14.355089 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:14.355090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:14.355101 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:14.355162 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:14.356296 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.356813 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.357826 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.358105 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.358184 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:14.360705 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:14.360707 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:14.360707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:14.360716 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:14.362412 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:14.362822 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:14.362823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:14.362824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:14.362905 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:14.363165 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:14.363166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:14.363166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:14.363332 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:14.363410 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:14.363666 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 16:36:14.363773 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:36:14.364045 8916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:14.354216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:33623 (local address 127.19.238.66:33389)
0812 16:36:14.354382 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:14.354425 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:14.354993 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:14.355239 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:14.355243 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:14.355945 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:14.356198 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:14.356204 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.356937 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.356938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:14.357729 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:14.357731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.358192 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.358193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:14.358360 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:14.358924 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:14.358935 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:14.360605 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:14.362512 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:14.362519 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:14.362521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:14.362743 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:14.362991 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:14.362992 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:14.362992 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:14.363094 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:14.363402 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:14.363406 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:14.363527 (+ 121us) client_negotiation.cc:769] Sending connection context
0812 16:36:14.363612 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 16:36:14.363738 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260812 16:36:14.364603 8914 heartbeater.cc:347] Connected to a master server at 127.19.238.126:33623
I20260812 16:36:14.364728 8914 heartbeater.cc:464] Registering TS with master...
I20260812 16:36:14.364950 8914 heartbeater.cc:511] Master 127.19.238.126:33623 requested a full tablet report, sending...
I20260812 16:36:14.365393 8522 ts_manager.cc:194] Registered new tserver with Master: 786834d7849e4759a008dd872d121f7d (127.19.238.66:41681)
I20260812 16:36:14.365824 8522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:33389
WARNING: no policy specified for kudu/127.19.238.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
I20260812 16:36:14.386863 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:33623
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:44541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:14.470583 8919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:14.470803 8919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:14.470827 8919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:14.471985 8919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:14.472028 8919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:14.472093 8919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:36:14.473578 8919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:44541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:33623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:14.473788 8919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:14.473968 8919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:14.479252 8927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:14.479292 8926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:14.479291 8929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:14.479269 8919 server_base.cc:1034] running on GCE node
I20260812 16:36:14.479789 8919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:14.480065 8919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:14.481210 8919 hybrid_clock.cc:648] HybridClock initialized: now 1786552574481191 us; error 40 us; skew 500 ppm
Aug 12 16:36:14 dist-test-slave-mvvj krb5kdc[8481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552574, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:14.484023 8919 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:36:14.484467 8919 webserver.cc:460] Webserver started at http://127.19.238.67:38423/ using document root <none> and password file <none>
I20260812 16:36:14.484704 8919 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:14.484772 8919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:14.484931 8919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:14.486023 8919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "c5f29952cdad40f182d417fb53e7f25c"
format_stamp: "Formatted at 2026-08-12 16:36:14 on dist-test-slave-mvvj"
server_key: "cf8de23fad86e1d637886e71e6b797f6"
server_key_iv: "b4b62c2e388d670a86cc817cd1ebf0ec"
server_key_version: "encryptionkey@0"
I20260812 16:36:14.486406 8919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "c5f29952cdad40f182d417fb53e7f25c"
format_stamp: "Formatted at 2026-08-12 16:36:14 on dist-test-slave-mvvj"
server_key: "cf8de23fad86e1d637886e71e6b797f6"
server_key_iv: "b4b62c2e388d670a86cc817cd1ebf0ec"
server_key_version: "encryptionkey@0"
I20260812 16:36:14.487780 8919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:36:14.488564 8938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:14.488729 8919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 16:36:14.488833 8919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "c5f29952cdad40f182d417fb53e7f25c"
format_stamp: "Formatted at 2026-08-12 16:36:14 on dist-test-slave-mvvj"
server_key: "cf8de23fad86e1d637886e71e6b797f6"
server_key_iv: "b4b62c2e388d670a86cc817cd1ebf0ec"
server_key_version: "encryptionkey@0"
I20260812 16:36:14.488920 8919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:14.507714 8919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:14.509663 8919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:14.509845 8919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:14.510118 8919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:36:14.510445 8919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:36:14.510505 8919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:14.510548 8919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:36:14.510594 8919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:14.517248 8919 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:33027
I20260812 16:36:14.517364 9075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:33027 every 8 connection(s)
I20260812 16:36:14.517660 8919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:36:14.522958 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 8919
I20260812 16:36:14.523072 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:36:14.523272 20409 external_mini_cluster.cc:1428] Setting key e5a7c81587accbfc1da2445bcc9dbddc
Aug 12 16:36:14 dist-test-slave-mvvj krb5kdc[8481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552574, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:36:14.527596 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:14.518485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:53075 (local address 127.19.238.126:33623)
0812 16:36:14.518549 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:14.518551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:14.518764 (+ 213us) server_negotiation.cc:407] Connection header received
0812 16:36:14.519349 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:14.519350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:14.519365 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:14.519424 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:14.520535 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.521077 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.522100 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.522303 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.522382 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:14.525004 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:14.525006 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:14.525006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:14.525013 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:14.526450 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:14.526800 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:14.526801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:14.526801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:14.526854 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:14.527057 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:14.527058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:14.527058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:14.527162 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:14.527212 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:14.527449 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 16:36:14.527530 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 16:36:14.527793 9078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:14.518458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:33623 (local address 127.19.238.67:53075)
0812 16:36:14.518642 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:14.518681 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:14.519227 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:14.519496 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:14.519500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:14.520230 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:14.520431 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:14.520436 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.521197 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.521198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:14.522003 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:14.522005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.522390 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.522391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:14.522524 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:14.523150 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:14.523160 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:14.524914 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:14.526524 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:14.526529 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:14.526531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:14.526735 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:14.526909 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:14.526910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:14.526910 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:14.527004 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:14.527217 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:14.527221 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:14.527337 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 16:36:14.527421 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 16:36:14.527549 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260812 16:36:14.528333 9076 heartbeater.cc:347] Connected to a master server at 127.19.238.126:33623
I20260812 16:36:14.528419 9076 heartbeater.cc:464] Registering TS with master...
I20260812 16:36:14.528568 9076 heartbeater.cc:511] Master 127.19.238.126:33623 requested a full tablet report, sending...
I20260812 16:36:14.528915 8522 ts_manager.cc:194] Registered new tserver with Master: c5f29952cdad40f182d417fb53e7f25c (127.19.238.67:33027)
I20260812 16:36:14.529253 8522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:53075
I20260812 16:36:14.535408 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:36:14.543718 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:14.536915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47338 (local address 127.19.238.126:33623)
0812 16:36:14.536977 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:14.536979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:14.537106 (+ 127us) server_negotiation.cc:407] Connection header received
0812 16:36:14.537147 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:14.537148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:14.537157 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:14.537213 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:14.537817 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.538323 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.539233 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.539415 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.539467 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:14.540901 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:14.540902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:14.540903 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:14.540909 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:14.542406 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:14.542822 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:14.542823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:14.542824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:14.542892 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:14.543151 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:14.543152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:14.543153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:14.543297 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:14.543400 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:14.543591 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 16:36:14.543650 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 16:36:14.545364 8522 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47338:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:36:14.547009 8522 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:36:14.554225 9090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:14.549817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:44331 (local address 127.0.0.1:43262)
0812 16:36:14.550202 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:14.550213 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:14.550290 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:14.550543 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:14.550544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:14.550547 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:14.550598 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:14.550600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.551644 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.551645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:14.553032 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:14.553035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.553903 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.553905 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:14.553983 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:14.553986 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:36:14.554054 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 16:36:14.554126 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":42,"thread_start_us":85,"threads_started":1}
I20260812 16:36:14.554270 9093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:14.550140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53600 (local address 127.19.238.66:41681)
0812 16:36:14.550331 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:14.550336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:14.550536 (+ 200us) server_negotiation.cc:407] Connection header received
0812 16:36:14.550583 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:14.550586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:14.550601 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:14.550674 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:14.550887 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.551936 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.553175 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.553822 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.553887 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:14.553947 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:14.554130 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 16:36:14.554175 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":82,"threads_started":1}
I20260812 16:36:14.554225 9092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:14.550027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:41681 (local address 127.0.0.1:53600)
0812 16:36:14.550406 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:14.550415 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:14.550459 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:14.550758 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:14.550759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:14.550761 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:14.550800 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:14.550802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.552036 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.552037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:14.553104 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:14.553107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.553980 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.553981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:14.554052 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:14.554056 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:36:14.554109 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 16:36:14.554140 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":79,"threads_started":1}
I20260812 16:36:14.554576 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:14.549937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43262 (local address 127.19.238.65:44331)
0812 16:36:14.550147 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:14.550151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:14.550263 (+ 112us) server_negotiation.cc:407] Connection header received
0812 16:36:14.550414 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:14.550418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:14.550436 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:14.550675 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:14.550756 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.551542 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.553124 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.553807 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.554104 (+ 297us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:14.554212 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:14.554237 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 16:36:14.554282 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":93,"threads_started":1}
I20260812 16:36:14.554595 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:14.549596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:33027 (local address 127.0.0.1:48204)
0812 16:36:14.549706 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:14.549748 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:14.549814 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:14.550193 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:14.550197 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:14.550200 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:14.550290 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:14.550292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.551659 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.551661 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:14.552881 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:14.552884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.553864 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.553866 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:14.553949 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:14.553964 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 16:36:14.554044 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 16:36:14.554520 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":64,"thread_start_us":61,"threads_started":1}
I20260812 16:36:14.554407 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:14.549696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48204 (local address 127.19.238.67:33027)
0812 16:36:14.550002 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:14.550009 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:14.550021 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:36:14.550066 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:14.550068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:14.550086 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:14.550185 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:14.550392 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.551542 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.552988 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.553759 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.553835 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:14.553902 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:14.554268 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 16:36:14.554319 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":127,"threads_started":1}
I20260812 16:36:14.555280 8838 tablet_service.cc:1467] Processing CreateTablet for tablet f560db4cf3a3493796712a4d9fa725ea (DEFAULT_TABLE table=test-table [id=c651addd770047b18b2a8730e7fb0606]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:36:14.555464 8679 tablet_service.cc:1467] Processing CreateTablet for tablet f560db4cf3a3493796712a4d9fa725ea (DEFAULT_TABLE table=test-table [id=c651addd770047b18b2a8730e7fb0606]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:36:14.555696 8838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f560db4cf3a3493796712a4d9fa725ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:14.555677 8999 tablet_service.cc:1467] Processing CreateTablet for tablet f560db4cf3a3493796712a4d9fa725ea (DEFAULT_TABLE table=test-table [id=c651addd770047b18b2a8730e7fb0606]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:36:14.555891 8679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f560db4cf3a3493796712a4d9fa725ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:14.556022 8999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f560db4cf3a3493796712a4d9fa725ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:14.558792 9094 tablet_bootstrap.cc:492] T f560db4cf3a3493796712a4d9fa725ea P ce8b41c300184958a65f16d118699094: Bootstrap starting.
I20260812 16:36:14.559224 9096 tablet_bootstrap.cc:492] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d: Bootstrap starting.
I20260812 16:36:14.559756 9094 tablet_bootstrap.cc:654] T f560db4cf3a3493796712a4d9fa725ea P ce8b41c300184958a65f16d118699094: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:14.559894 9096 tablet_bootstrap.cc:654] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:14.560065 9095 tablet_bootstrap.cc:492] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c: Bootstrap starting.
I20260812 16:36:14.560447 9094 log.cc:824] T f560db4cf3a3493796712a4d9fa725ea P ce8b41c300184958a65f16d118699094: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:14.560504 9096 log.cc:824] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:14.560705 9095 tablet_bootstrap.cc:654] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:14.561280 9094 tablet_bootstrap.cc:492] T f560db4cf3a3493796712a4d9fa725ea P ce8b41c300184958a65f16d118699094: No bootstrap required, opened a new log
I20260812 16:36:14.561277 9095 log.cc:824] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:14.561369 9094 ts_tablet_manager.cc:1397] T f560db4cf3a3493796712a4d9fa725ea P ce8b41c300184958a65f16d118699094: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:36:14.561576 9096 tablet_bootstrap.cc:492] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d: No bootstrap required, opened a new log
I20260812 16:36:14.561643 9096 ts_tablet_manager.cc:1397] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:36:14.562016 9095 tablet_bootstrap.cc:492] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c: No bootstrap required, opened a new log
I20260812 16:36:14.562099 9095 ts_tablet_manager.cc:1397] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:36:14.563455 9094 raft_consensus.cc:348] T f560db4cf3a3493796712a4d9fa725ea P ce8b41c300184958a65f16d118699094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f29952cdad40f182d417fb53e7f25c" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33027 } } peers { permanent_uuid: "ce8b41c300184958a65f16d118699094" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44331 } } peers { permanent_uuid: "786834d7849e4759a008dd872d121f7d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41681 } }
I20260812 16:36:14.563665 9094 raft_consensus.cc:374] T f560db4cf3a3493796712a4d9fa725ea P ce8b41c300184958a65f16d118699094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:14.563586 9096 raft_consensus.cc:348] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f29952cdad40f182d417fb53e7f25c" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33027 } } peers { permanent_uuid: "ce8b41c300184958a65f16d118699094" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44331 } } peers { permanent_uuid: "786834d7849e4759a008dd872d121f7d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41681 } }
I20260812 16:36:14.563707 9094 raft_consensus.cc:729] T f560db4cf3a3493796712a4d9fa725ea P ce8b41c300184958a65f16d118699094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce8b41c300184958a65f16d118699094, State: Initialized, Role: FOLLOWER
I20260812 16:36:14.563735 9096 raft_consensus.cc:374] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:14.563768 9096 raft_consensus.cc:729] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 786834d7849e4759a008dd872d121f7d, State: Initialized, Role: FOLLOWER
I20260812 16:36:14.563619 9095 raft_consensus.cc:348] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f29952cdad40f182d417fb53e7f25c" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33027 } } peers { permanent_uuid: "ce8b41c300184958a65f16d118699094" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44331 } } peers { permanent_uuid: "786834d7849e4759a008dd872d121f7d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41681 } }
I20260812 16:36:14.563807 9095 raft_consensus.cc:374] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:14.563870 9095 raft_consensus.cc:729] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5f29952cdad40f182d417fb53e7f25c, State: Initialized, Role: FOLLOWER
I20260812 16:36:14.563898 9094 consensus_queue.cc:260] T f560db4cf3a3493796712a4d9fa725ea P ce8b41c300184958a65f16d118699094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f29952cdad40f182d417fb53e7f25c" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33027 } } peers { permanent_uuid: "ce8b41c300184958a65f16d118699094" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44331 } } peers { permanent_uuid: "786834d7849e4759a008dd872d121f7d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41681 } }
I20260812 16:36:14.563931 9096 consensus_queue.cc:260] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f29952cdad40f182d417fb53e7f25c" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33027 } } peers { permanent_uuid: "ce8b41c300184958a65f16d118699094" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44331 } } peers { permanent_uuid: "786834d7849e4759a008dd872d121f7d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41681 } }
I20260812 16:36:14.563984 9095 consensus_queue.cc:260] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f29952cdad40f182d417fb53e7f25c" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33027 } } peers { permanent_uuid: "ce8b41c300184958a65f16d118699094" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44331 } } peers { permanent_uuid: "786834d7849e4759a008dd872d121f7d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41681 } }
I20260812 16:36:14.564213 9096 ts_tablet_manager.cc:1428] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 16:36:14.564213 9094 ts_tablet_manager.cc:1428] T f560db4cf3a3493796712a4d9fa725ea P ce8b41c300184958a65f16d118699094: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:36:14.564319 9095 ts_tablet_manager.cc:1428] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 16:36:14.564342 8914 heartbeater.cc:503] Master 127.19.238.126:33623 was elected leader, sending a full tablet report...
I20260812 16:36:14.564350 8755 heartbeater.cc:503] Master 127.19.238.126:33623 was elected leader, sending a full tablet report...
I20260812 16:36:14.564604 9076 heartbeater.cc:503] Master 127.19.238.126:33623 was elected leader, sending a full tablet report...
W20260812 16:36:14.604233 8915 tablet.cc:2369] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:36:14.678849 8756 tablet.cc:2369] T f560db4cf3a3493796712a4d9fa725ea P ce8b41c300184958a65f16d118699094: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:36:14.768268 9077 tablet.cc:2369] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:36:14.787035 9103 raft_consensus.cc:482] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:36:14.787169 9103 raft_consensus.cc:504] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f29952cdad40f182d417fb53e7f25c" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33027 } } peers { permanent_uuid: "ce8b41c300184958a65f16d118699094" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44331 } } peers { permanent_uuid: "786834d7849e4759a008dd872d121f7d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41681 } }
I20260812 16:36:14.787477 9103 leader_election.cc:288] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce8b41c300184958a65f16d118699094 (127.19.238.65:44331), 786834d7849e4759a008dd872d121f7d (127.19.238.66:41681)
I20260812 16:36:14.791055 9078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:14.787777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:44331 (local address 127.19.238.67:53377)
0812 16:36:14.787867 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:14.787880 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:14.787929 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:14.788101 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:14.788102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:14.788104 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:14.788188 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:14.788190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.788970 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.788971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:14.789835 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:14.789838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.790756 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.790757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:14.790840 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:14.790844 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:36:14.790912 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 16:36:14.790952 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":6}
I20260812 16:36:14.791088 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:14.787870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:53377 (local address 127.19.238.65:44331)
0812 16:36:14.787938 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:14.787941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:14.787959 (+ 18us) server_negotiation.cc:407] Connection header received
0812 16:36:14.787994 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:14.787995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:14.788009 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:14.788084 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:14.788273 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.788887 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.789958 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.790575 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.790634 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:14.790643 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:14.790953 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 16:36:14.791023 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:36:14.791410 8702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f560db4cf3a3493796712a4d9fa725ea" candidate_uuid: "c5f29952cdad40f182d417fb53e7f25c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce8b41c300184958a65f16d118699094" is_pre_election: true
I20260812 16:36:14.791599 8702 raft_consensus.cc:2454] T f560db4cf3a3493796712a4d9fa725ea P ce8b41c300184958a65f16d118699094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5f29952cdad40f182d417fb53e7f25c in term 0.
I20260812 16:36:14.791625 9104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:14.787949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:41681 (local address 127.19.238.67:45933)
0812 16:36:14.788174 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:14.788186 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:14.788267 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:14.788410 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:14.788411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:14.788412 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:14.788479 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:14.788481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.789158 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.789159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:14.790339 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:14.790342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.791352 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.791353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:14.791420 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:14.791426 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 16:36:14.791511 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 16:36:14.791551 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":148,"threads_started":1}
I20260812 16:36:14.791631 9093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:14.788041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:45933 (local address 127.19.238.66:41681)
0812 16:36:14.788091 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:14.788093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:14.788282 (+ 189us) server_negotiation.cc:407] Connection header received
0812 16:36:14.788331 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:14.788332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:14.788344 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:14.788389 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:14.788552 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.789077 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.790531 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.791281 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.791350 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:14.791360 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:14.791529 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 16:36:14.791581 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 16:36:14.791904 8942 leader_election.cc:302] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5f29952cdad40f182d417fb53e7f25c, ce8b41c300184958a65f16d118699094; no voters:
I20260812 16:36:14.791978 8862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f560db4cf3a3493796712a4d9fa725ea" candidate_uuid: "c5f29952cdad40f182d417fb53e7f25c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "786834d7849e4759a008dd872d121f7d" is_pre_election: true
I20260812 16:36:14.792053 9103 raft_consensus.cc:2781] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:36:14.792132 9103 raft_consensus.cc:482] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:36:14.792156 9103 raft_consensus.cc:3037] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:14.792160 8862 raft_consensus.cc:2454] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5f29952cdad40f182d417fb53e7f25c in term 0.
I20260812 16:36:14.792976 9103 raft_consensus.cc:504] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f29952cdad40f182d417fb53e7f25c" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33027 } } peers { permanent_uuid: "ce8b41c300184958a65f16d118699094" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44331 } } peers { permanent_uuid: "786834d7849e4759a008dd872d121f7d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41681 } }
I20260812 16:36:14.793123 9103 leader_election.cc:288] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [CANDIDATE]: Term 1 election: Requested vote from peers ce8b41c300184958a65f16d118699094 (127.19.238.65:44331), 786834d7849e4759a008dd872d121f7d (127.19.238.66:41681)
I20260812 16:36:14.793356 8862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f560db4cf3a3493796712a4d9fa725ea" candidate_uuid: "c5f29952cdad40f182d417fb53e7f25c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "786834d7849e4759a008dd872d121f7d"
I20260812 16:36:14.793430 8862 raft_consensus.cc:3037] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:14.793324 8702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f560db4cf3a3493796712a4d9fa725ea" candidate_uuid: "c5f29952cdad40f182d417fb53e7f25c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce8b41c300184958a65f16d118699094"
I20260812 16:36:14.793509 8702 raft_consensus.cc:3037] T f560db4cf3a3493796712a4d9fa725ea P ce8b41c300184958a65f16d118699094 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:14.794273 8862 raft_consensus.cc:2454] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5f29952cdad40f182d417fb53e7f25c in term 1.
I20260812 16:36:14.794384 8702 raft_consensus.cc:2454] T f560db4cf3a3493796712a4d9fa725ea P ce8b41c300184958a65f16d118699094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5f29952cdad40f182d417fb53e7f25c in term 1.
I20260812 16:36:14.794469 8942 leader_election.cc:302] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786834d7849e4759a008dd872d121f7d, c5f29952cdad40f182d417fb53e7f25c; no voters:
I20260812 16:36:14.794621 9103 raft_consensus.cc:2781] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:14.794824 9103 raft_consensus.cc:686] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [term 1 LEADER]: Becoming Leader. State: Replica: c5f29952cdad40f182d417fb53e7f25c, State: Running, Role: LEADER
I20260812 16:36:14.794936 9103 consensus_queue.cc:237] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f29952cdad40f182d417fb53e7f25c" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33027 } } peers { permanent_uuid: "ce8b41c300184958a65f16d118699094" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44331 } } peers { permanent_uuid: "786834d7849e4759a008dd872d121f7d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41681 } }
I20260812 16:36:14.795933 8531 catalog_manager.cc:5495] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c reported cstate change: term changed from 0 to 1, leader changed from <none> to c5f29952cdad40f182d417fb53e7f25c (127.19.238.67). New cstate: current_term: 1 leader_uuid: "c5f29952cdad40f182d417fb53e7f25c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f29952cdad40f182d417fb53e7f25c" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 33027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce8b41c300184958a65f16d118699094" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786834d7849e4759a008dd872d121f7d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41681 } health_report { overall_health: UNKNOWN } } }
I20260812 16:36:14.844311 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:14.841681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48208 (local address 127.19.238.67:33027)
0812 16:36:14.841729 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:14.841731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:14.841740 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:36:14.841822 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:14.841823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:14.841842 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:14.841900 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:36:14.842083 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.842636 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.843450 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:14.843645 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:14.843710 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:14.843899 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:36:14.844009 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:36:14.844138 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:14.844193 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 16:36:14.844205 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 16:36:14.846374 8702 raft_consensus.cc:1261] T f560db4cf3a3493796712a4d9fa725ea P ce8b41c300184958a65f16d118699094 [term 1 FOLLOWER]: Refusing update from remote peer c5f29952cdad40f182d417fb53e7f25c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:36:14.846374 8862 raft_consensus.cc:1261] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d [term 1 FOLLOWER]: Refusing update from remote peer c5f29952cdad40f182d417fb53e7f25c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:36:14.846810 9105 consensus_queue.cc:1035] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [LEADER]: Connected to new peer: Peer: permanent_uuid: "786834d7849e4759a008dd872d121f7d" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:36:14.846976 9103 consensus_queue.cc:1035] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce8b41c300184958a65f16d118699094" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 44331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:36:14.849779 9109 mvcc.cc:205] Tried to move back new op lower bound from 7317719346567483392 to 7317719346360647680. Current Snapshot: MvccSnapshot[applied={T|T < 7317719346567483392}]
I20260812 16:36:14.851039 9110 mvcc.cc:205] Tried to move back new op lower bound from 7317719346567483392 to 7317719346360647680. Current Snapshot: MvccSnapshot[applied={T|T < 7317719346567483392 or (T in {7317719346567483392})}]
I20260812 16:36:14.852855 8530 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:36:14.852970 8530 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:36:14.853832 8530 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a437a4e5ba247349dd5d4c51265f74e: Sending DeleteTablet for 3 replicas of tablet f560db4cf3a3493796712a4d9fa725ea
I20260812 16:36:14.854242 8679 tablet_service.cc:1514] Processing DeleteTablet for tablet f560db4cf3a3493796712a4d9fa725ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:36:14 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:43262
I20260812 16:36:14.854255 8999 tablet_service.cc:1514] Processing DeleteTablet for tablet f560db4cf3a3493796712a4d9fa725ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:36:14 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:48204
I20260812 16:36:14.854575 9120 tablet_replica.cc:330] T f560db4cf3a3493796712a4d9fa725ea P ce8b41c300184958a65f16d118699094: stopping tablet replica
I20260812 16:36:14.854635 8838 tablet_service.cc:1514] Processing DeleteTablet for tablet f560db4cf3a3493796712a4d9fa725ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:36:14 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:53600
I20260812 16:36:14.854697 9120 raft_consensus.cc:2229] T f560db4cf3a3493796712a4d9fa725ea P ce8b41c300184958a65f16d118699094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:36:14.854838 9122 tablet_replica.cc:330] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d: stopping tablet replica
I20260812 16:36:14.854820 9120 raft_consensus.cc:2258] T f560db4cf3a3493796712a4d9fa725ea P ce8b41c300184958a65f16d118699094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:36:14.854962 9122 raft_consensus.cc:2229] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:36:14.855068 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 8604
I20260812 16:36:14.855090 9122 raft_consensus.cc:2258] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:36:14.855369 9120 ts_tablet_manager.cc:1905] T f560db4cf3a3493796712a4d9fa725ea P ce8b41c300184958a65f16d118699094: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:36:14.855540 9122 ts_tablet_manager.cc:1905] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:36:14.855739 9121 tablet_replica.cc:330] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c: stopping tablet replica
I20260812 16:36:14.855863 9121 raft_consensus.cc:2229] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:36:14.856009 9121 raft_consensus.cc:2258] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:36:14.857525 9121 ts_tablet_manager.cc:1905] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:36:14.857547 9122 ts_tablet_manager.cc:1918] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:36:14.857614 9122 log.cc:1196] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/f560db4cf3a3493796712a4d9fa725ea
I20260812 16:36:14.857831 9122 ts_tablet_manager.cc:1939] T f560db4cf3a3493796712a4d9fa725ea P 786834d7849e4759a008dd872d121f7d: Deleting consensus metadata
I20260812 16:36:14.858208 8516 catalog_manager.cc:4841] TS 786834d7849e4759a008dd872d121f7d (127.19.238.66:41681): tablet f560db4cf3a3493796712a4d9fa725ea (table test-table [id=c651addd770047b18b2a8730e7fb0606]) successfully deleted
I20260812 16:36:14.859864 9121 ts_tablet_manager.cc:1918] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:36:14.859935 9121 log.cc:1196] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/f560db4cf3a3493796712a4d9fa725ea
I20260812 16:36:14.860201 9121 ts_tablet_manager.cc:1939] T f560db4cf3a3493796712a4d9fa725ea P c5f29952cdad40f182d417fb53e7f25c: Deleting consensus metadata
I20260812 16:36:14.860761 8516 catalog_manager.cc:4841] TS c5f29952cdad40f182d417fb53e7f25c (127.19.238.67:33027): tablet f560db4cf3a3493796712a4d9fa725ea (table test-table [id=c651addd770047b18b2a8730e7fb0606]) successfully deleted
W20260812 16:36:14.866176 8516 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.238.65:44331 (error 108)
I20260812 16:36:14.866779 9092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:14.866637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:44331 (local address 127.0.0.1:43270)
0812 16:36:14.866695 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:14.866735 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 16:36:14.866896 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 8760
W20260812 16:36:14.869921 8516 catalog_manager.cc:4584] TS ce8b41c300184958a65f16d118699094 (127.19.238.65:44331): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f560db4cf3a3493796712a4d9fa725ea: Network error: Client connection negotiation failed: client connection to 127.19.238.65:44331: connect: Connection refused (error 111)
I20260812 16:36:14.874231 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 8919
I20260812 16:36:14.882441 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 8495
2026-08-12T16:36:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2284 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T16:36:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:14Z Disabled control of system clock
I20260812 16:36:14.914119 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:33149
--webserver_interface=127.19.238.126
--webserver_port=0
--builtin_ntp_servers=127.19.238.84:34435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:33149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:36:14.997362 9128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:14.997594 9128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:14.997615 9128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:14.998862 9128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:14.998904 9128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:14.998917 9128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:14.998930 9128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:14.998942 9128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:15.000371 9128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:34435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:33149
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:33149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:15.000636 9128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:15.000936 9128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:15.005764 9139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:15.005757 9136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:15.005916 9137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:15.006258 9128 server_base.cc:1034] running on GCE node
I20260812 16:36:15.006541 9128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:15.006861 9128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:15.008065 9128 hybrid_clock.cc:648] HybridClock initialized: now 1786552575008040 us; error 34 us; skew 500 ppm
I20260812 16:36:15.009351 9128 webserver.cc:460] Webserver started at http://127.19.238.126:41367/ using document root <none> and password file <none>
I20260812 16:36:15.009609 9128 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:15.009681 9128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:15.009843 9128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:15.011032 9128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "e63ac311f1174aa0b3f8adfe4a286a33"
format_stamp: "Formatted at 2026-08-12 16:36:15 on dist-test-slave-mvvj"
server_key: "9d4e7d37582bc4dbd3242f657c7c3e08"
server_key_iv: "38a872cdb7d1c51519c53c6424dfbeda"
server_key_version: "encryptionkey@0"
I20260812 16:36:15.011418 9128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "e63ac311f1174aa0b3f8adfe4a286a33"
format_stamp: "Formatted at 2026-08-12 16:36:15 on dist-test-slave-mvvj"
server_key: "9d4e7d37582bc4dbd3242f657c7c3e08"
server_key_iv: "38a872cdb7d1c51519c53c6424dfbeda"
server_key_version: "encryptionkey@0"
I20260812 16:36:15.012816 9128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:36:15.013717 9145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:15.013886 9128 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:15.014005 9128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "e63ac311f1174aa0b3f8adfe4a286a33"
format_stamp: "Formatted at 2026-08-12 16:36:15 on dist-test-slave-mvvj"
server_key: "9d4e7d37582bc4dbd3242f657c7c3e08"
server_key_iv: "38a872cdb7d1c51519c53c6424dfbeda"
server_key_version: "encryptionkey@0"
I20260812 16:36:15.014086 9128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:15.027397 9128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:15.027753 9128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:15.027941 9128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:15.031762 9128 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:33149
I20260812 16:36:15.031801 9204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:33149 every 8 connection(s)
I20260812 16:36:15.032219 9128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:15.033233 9205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:15.035581 9205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33: Bootstrap starting.
I20260812 16:36:15.036228 9205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:15.036545 9205 log.cc:824] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:15.037241 9205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33: No bootstrap required, opened a new log
I20260812 16:36:15.038856 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 9128
I20260812 16:36:15.038968 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:15.039023 9205 raft_consensus.cc:348] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63ac311f1174aa0b3f8adfe4a286a33" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 33149 } }
I20260812 16:36:15.039192 20409 external_mini_cluster.cc:1428] Setting key b764571d7201eef1f90e054f56561422
I20260812 16:36:15.039206 9205 raft_consensus.cc:374] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:15.039245 9205 raft_consensus.cc:729] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63ac311f1174aa0b3f8adfe4a286a33, State: Initialized, Role: FOLLOWER
I20260812 16:36:15.039397 9205 consensus_queue.cc:260] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63ac311f1174aa0b3f8adfe4a286a33" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 33149 } }
I20260812 16:36:15.039487 9205 raft_consensus.cc:388] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:15.039534 9205 raft_consensus.cc:482] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:15.039656 9205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:15.040429 9205 raft_consensus.cc:504] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63ac311f1174aa0b3f8adfe4a286a33" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 33149 } }
I20260812 16:36:15.040573 9205 leader_election.cc:302] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e63ac311f1174aa0b3f8adfe4a286a33; no voters:
I20260812 16:36:15.040787 9205 leader_election.cc:288] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:15.040817 9210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:15.040930 9210 raft_consensus.cc:686] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33 [term 1 LEADER]: Becoming Leader. State: Replica: e63ac311f1174aa0b3f8adfe4a286a33, State: Running, Role: LEADER
I20260812 16:36:15.041077 9210 consensus_queue.cc:237] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63ac311f1174aa0b3f8adfe4a286a33" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 33149 } }
I20260812 16:36:15.041064 9205 sys_catalog.cc:564] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:15.041617 9212 sys_catalog.cc:455] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e63ac311f1174aa0b3f8adfe4a286a33. Latest consensus state: current_term: 1 leader_uuid: "e63ac311f1174aa0b3f8adfe4a286a33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63ac311f1174aa0b3f8adfe4a286a33" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 33149 } } }
I20260812 16:36:15.041697 9212 sys_catalog.cc:458] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:15.041937 9211 sys_catalog.cc:455] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e63ac311f1174aa0b3f8adfe4a286a33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63ac311f1174aa0b3f8adfe4a286a33" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 33149 } } }
I20260812 16:36:15.042083 9211 sys_catalog.cc:458] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:15.042249 9218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:15.042786 9218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:15.044380 9218 catalog_manager.cc:1347] Generated new cluster ID: c2e1bc672201471ab6c13a2cfab2ee8f
I20260812 16:36:15.044438 9218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:15.045678 9209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.039718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33130 (local address 127.19.238.126:33149)
0812 16:36:15.039990 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:15.039999 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:15.040040 (+ 41us) server_negotiation.cc:407] Connection header received
0812 16:36:15.040163 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:15.040171 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:15.040251 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:15.040401 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:15.040624 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.042499 (+ 1875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.043381 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.044147 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.044285 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:15.045046 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:15.045050 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:15.045050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:15.045227 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:15.045260 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:15.045271 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:15.045346 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:15.045422 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 16:36:15.045502 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":156,"threads_started":1}
I20260812 16:36:15.053855 9218 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:15.054687 9218 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:15.063530 9218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33: Generated new TSK 0
I20260812 16:36:15.063885 9218 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:15.066730 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:33149
--builtin_ntp_servers=127.19.238.84:34435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:36:15.152601 9232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:15.152841 9232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:15.152877 9232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:15.154017 9232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:15.154083 9232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:15.154126 9232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:36:15.155519 9232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:34435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:33149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:15.155789 9232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:15.156128 9232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:15.161880 9238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:15.161918 9239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:15.161899 9241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:15.162492 9232 server_base.cc:1034] running on GCE node
I20260812 16:36:15.162732 9232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:15.163030 9232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:15.164197 9232 hybrid_clock.cc:648] HybridClock initialized: now 1786552575164172 us; error 41 us; skew 500 ppm
I20260812 16:36:15.165812 9232 webserver.cc:460] Webserver started at http://127.19.238.65:44423/ using document root <none> and password file <none>
I20260812 16:36:15.166128 9232 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:15.166241 9232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:15.166438 9232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:15.167778 9232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "cea417aa588d43eda5f718342f43424f"
format_stamp: "Formatted at 2026-08-12 16:36:15 on dist-test-slave-mvvj"
server_key: "eddebe8b12f837317059a4a1fbf8da69"
server_key_iv: "8c01854d80482cccce49e2049c4d156b"
server_key_version: "encryptionkey@0"
I20260812 16:36:15.168253 9232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "cea417aa588d43eda5f718342f43424f"
format_stamp: "Formatted at 2026-08-12 16:36:15 on dist-test-slave-mvvj"
server_key: "eddebe8b12f837317059a4a1fbf8da69"
server_key_iv: "8c01854d80482cccce49e2049c4d156b"
server_key_version: "encryptionkey@0"
I20260812 16:36:15.169746 9232 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:36:15.170614 9248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:15.170786 9232 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:15.170887 9232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "cea417aa588d43eda5f718342f43424f"
format_stamp: "Formatted at 2026-08-12 16:36:15 on dist-test-slave-mvvj"
server_key: "eddebe8b12f837317059a4a1fbf8da69"
server_key_iv: "8c01854d80482cccce49e2049c4d156b"
server_key_version: "encryptionkey@0"
I20260812 16:36:15.170969 9232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:15.180130 9232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:15.180461 9232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:15.180624 9232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:15.180863 9232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:36:15.181229 9232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:36:15.181275 9232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:15.181298 9232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:36:15.181351 9232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:15.187902 9232 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:41167
I20260812 16:36:15.188006 9376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:41167 every 8 connection(s)
I20260812 16:36:15.188320 9232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:36:15.192785 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 9232
I20260812 16:36:15.192924 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:36:15.193161 20409 external_mini_cluster.cc:1428] Setting key c7f494a138d21d1b5a738e8bd1d2f043
I20260812 16:36:15.194439 9209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.189108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:36055 (local address 127.19.238.126:33149)
0812 16:36:15.189187 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:15.189189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:15.189422 (+ 233us) server_negotiation.cc:407] Connection header received
0812 16:36:15.190036 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:15.190038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:15.190052 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:15.190127 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:15.190862 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.191414 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.192609 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.192952 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.193049 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:15.194084 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:15.194087 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:15.194087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:15.194093 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:15.194115 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:15.194120 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:15.194178 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:15.194284 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 16:36:15.194323 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 16:36:15.194449 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:33149
--builtin_ntp_servers=127.19.238.84:34435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 16:36:15.194615 9380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.189089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:33149 (local address 127.19.238.65:36055)
0812 16:36:15.189287 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:15.189334 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:15.189907 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:15.190210 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:15.190214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:15.190223 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:15.190769 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:15.190773 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.191519 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.191520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:15.192494 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:15.192497 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.193053 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.193054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:15.193175 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:15.193884 (+ 709us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:36:15.193884 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:15.193936 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:36:15.193967 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:15.194191 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:15.194197 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:15.194199 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 16:36:15.194260 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 16:36:15.194349 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260812 16:36:15.195181 9378 heartbeater.cc:347] Connected to a master server at 127.19.238.126:33149
I20260812 16:36:15.195287 9378 heartbeater.cc:464] Registering TS with master...
I20260812 16:36:15.195494 9378 heartbeater.cc:511] Master 127.19.238.126:33149 requested a full tablet report, sending...
I20260812 16:36:15.196105 9165 ts_manager.cc:194] Registered new tserver with Master: cea417aa588d43eda5f718342f43424f (127.19.238.65:41167)
I20260812 16:36:15.197184 9165 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.238.65:36055
W20260812 16:36:15.280560 9381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:15.280772 9381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:15.280794 9381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:15.281877 9381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:15.281921 9381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:15.281948 9381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:36:15.283253 9381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:34435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:33149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:15.283458 9381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:15.283704 9381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:15.289198 9387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:15.289208 9388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:15.289165 9390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:15.289577 9381 server_base.cc:1034] running on GCE node
I20260812 16:36:15.289812 9381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:15.290107 9381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:15.291282 9381 hybrid_clock.cc:648] HybridClock initialized: now 1786552575291239 us; error 86 us; skew 500 ppm
I20260812 16:36:15.292824 9381 webserver.cc:460] Webserver started at http://127.19.238.66:43491/ using document root <none> and password file <none>
I20260812 16:36:15.293128 9381 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:15.293207 9381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:15.293368 9381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:15.294519 9381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "1c00ec10a4f2472980667ae5f5417116"
format_stamp: "Formatted at 2026-08-12 16:36:15 on dist-test-slave-mvvj"
server_key: "abd629f0da529ac933c60f479ea29294"
server_key_iv: "e3b89db1c26c90d232a16a5c816c115a"
server_key_version: "encryptionkey@0"
I20260812 16:36:15.294920 9381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "1c00ec10a4f2472980667ae5f5417116"
format_stamp: "Formatted at 2026-08-12 16:36:15 on dist-test-slave-mvvj"
server_key: "abd629f0da529ac933c60f479ea29294"
server_key_iv: "e3b89db1c26c90d232a16a5c816c115a"
server_key_version: "encryptionkey@0"
I20260812 16:36:15.296532 9381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:36:15.297472 9398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:15.297680 9381 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:15.297794 9381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "1c00ec10a4f2472980667ae5f5417116"
format_stamp: "Formatted at 2026-08-12 16:36:15 on dist-test-slave-mvvj"
server_key: "abd629f0da529ac933c60f479ea29294"
server_key_iv: "e3b89db1c26c90d232a16a5c816c115a"
server_key_version: "encryptionkey@0"
I20260812 16:36:15.297883 9381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:15.307297 9381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:15.307610 9381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:15.307761 9381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:15.308043 9381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:36:15.308377 9381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:36:15.308434 9381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:15.308476 9381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:36:15.308501 9381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:15.314615 9381 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:44451
I20260812 16:36:15.314688 9530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:44451 every 8 connection(s)
I20260812 16:36:15.315093 9381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:36:15.320698 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 9381
I20260812 16:36:15.320830 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:36:15.320837 9209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.315988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:60549 (local address 127.19.238.126:33149)
0812 16:36:15.316102 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:15.316104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:15.316332 (+ 228us) server_negotiation.cc:407] Connection header received
0812 16:36:15.316945 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:15.316947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:15.316959 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:15.317022 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:15.317740 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.318252 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.319292 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.319500 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.319607 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:15.320527 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:15.320529 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:15.320529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:15.320535 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:15.320550 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:15.320555 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:15.320627 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:15.320726 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 16:36:15.320756 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 16:36:15.321092 9533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.315937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:33149 (local address 127.19.238.66:60549)
0812 16:36:15.316200 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:15.316248 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:15.316812 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:15.317101 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:15.317104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:15.317112 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:15.317644 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:15.317649 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.318388 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.318389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:15.319182 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:15.319184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.319619 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.319620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:15.319761 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:15.320354 (+ 593us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:36:15.320354 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:15.320393 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:36:15.320419 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:15.320628 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:15.320635 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:15.320639 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:36:15.320731 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 16:36:15.320843 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":103,"threads_started":1}
I20260812 16:36:15.321094 20409 external_mini_cluster.cc:1428] Setting key 81fc03daf078b0e319ec256db488b8be
I20260812 16:36:15.321684 9531 heartbeater.cc:347] Connected to a master server at 127.19.238.126:33149
I20260812 16:36:15.321771 9531 heartbeater.cc:464] Registering TS with master...
I20260812 16:36:15.321988 9531 heartbeater.cc:511] Master 127.19.238.126:33149 requested a full tablet report, sending...
I20260812 16:36:15.322331 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:33149
--builtin_ntp_servers=127.19.238.84:34435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 16:36:15.322379 9164 ts_manager.cc:194] Registered new tserver with Master: 1c00ec10a4f2472980667ae5f5417116 (127.19.238.66:44451)
I20260812 16:36:15.322891 9164 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.238.66:60549
W20260812 16:36:15.406271 9534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:15.406481 9534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:15.406512 9534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:15.407648 9534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:15.407685 9534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:15.407711 9534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:36:15.409093 9534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:34435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:33149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:15.409287 9534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:15.409534 9534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:15.414870 9540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:15.415012 9534 server_base.cc:1034] running on GCE node
W20260812 16:36:15.414847 9541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:15.414913 9545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:15.415287 9534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:15.415563 9534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:15.416718 9534 hybrid_clock.cc:648] HybridClock initialized: now 1786552575416700 us; error 25 us; skew 500 ppm
I20260812 16:36:15.418113 9534 webserver.cc:460] Webserver started at http://127.19.238.67:37691/ using document root <none> and password file <none>
I20260812 16:36:15.418394 9534 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:15.418474 9534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:15.418637 9534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:15.419921 9534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "66e01960704542a5abbde7e9b7f8eac9"
format_stamp: "Formatted at 2026-08-12 16:36:15 on dist-test-slave-mvvj"
server_key: "64dd5d0148557d734399ddb59b323355"
server_key_iv: "4e6e908799c37378ff5ffb1c6d2f62a2"
server_key_version: "encryptionkey@0"
I20260812 16:36:15.420301 9534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "66e01960704542a5abbde7e9b7f8eac9"
format_stamp: "Formatted at 2026-08-12 16:36:15 on dist-test-slave-mvvj"
server_key: "64dd5d0148557d734399ddb59b323355"
server_key_iv: "4e6e908799c37378ff5ffb1c6d2f62a2"
server_key_version: "encryptionkey@0"
I20260812 16:36:15.421808 9534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:36:15.422784 9551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:15.422999 9534 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:36:15.423121 9534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "66e01960704542a5abbde7e9b7f8eac9"
format_stamp: "Formatted at 2026-08-12 16:36:15 on dist-test-slave-mvvj"
server_key: "64dd5d0148557d734399ddb59b323355"
server_key_iv: "4e6e908799c37378ff5ffb1c6d2f62a2"
server_key_version: "encryptionkey@0"
I20260812 16:36:15.423208 9534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:15.445371 9534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:15.445768 9534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:15.445945 9534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:15.446218 9534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:36:15.446610 9534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:36:15.446667 9534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:15.446720 9534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:36:15.446768 9534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:15.452775 9534 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:40735
I20260812 16:36:15.452857 9687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:40735 every 8 connection(s)
I20260812 16:36:15.453241 9534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:36:15.458611 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 9534
I20260812 16:36:15.458736 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:36:15.458994 20409 external_mini_cluster.cc:1428] Setting key 4ef7772b627f575969b3f79fb118197f
I20260812 16:36:15.460629 9209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.454110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:56451 (local address 127.19.238.126:33149)
0812 16:36:15.454184 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:15.454187 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:15.454474 (+ 287us) server_negotiation.cc:407] Connection header received
0812 16:36:15.454981 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:15.454982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:15.454994 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:15.455054 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:15.455939 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.457027 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.458703 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.459087 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.459171 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:15.460294 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:15.460296 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:15.460296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:15.460309 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:15.460326 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:15.460331 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:15.460382 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:15.460498 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 16:36:15.460533 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 16:36:15.460892 9690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.454088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:33149 (local address 127.19.238.67:56451)
0812 16:36:15.454318 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:15.454367 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:15.454874 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:15.455113 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:15.455116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:15.455124 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:15.455775 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:15.455792 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.457149 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.457150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:15.458582 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:15.458586 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.459196 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.459197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:15.459426 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:15.460115 (+ 689us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:36:15.460116 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:15.460158 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:36:15.460193 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:15.460405 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:15.460412 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:15.460415 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:36:15.460490 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 16:36:15.460599 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":102,"threads_started":1}
I20260812 16:36:15.461503 9688 heartbeater.cc:347] Connected to a master server at 127.19.238.126:33149
I20260812 16:36:15.461620 9688 heartbeater.cc:464] Registering TS with master...
I20260812 16:36:15.461840 9688 heartbeater.cc:511] Master 127.19.238.126:33149 requested a full tablet report, sending...
I20260812 16:36:15.462276 9164 ts_manager.cc:194] Registered new tserver with Master: 66e01960704542a5abbde7e9b7f8eac9 (127.19.238.67:40735)
I20260812 16:36:15.462832 9164 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.238.67:56451
I20260812 16:36:15.470943 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:36:15.476022 9209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.472632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33140 (local address 127.19.238.126:33149)
0812 16:36:15.472710 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:15.472712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:15.472776 (+ 64us) server_negotiation.cc:407] Connection header received
0812 16:36:15.472832 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:15.472832 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:15.472843 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:15.472889 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:15.473030 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.473600 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.474499 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.474728 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.474825 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:15.475611 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:15.475613 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:15.475613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:15.475618 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:15.475637 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:15.475642 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:15.475762 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:15.475917 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 16:36:15.475930 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 16:36:15.477655 9164 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:36:15.479173 9164 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:36:15.486099 9696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.481674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:44451 (local address 127.0.0.1:41094)
0812 16:36:15.482045 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:15.482087 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:15.482161 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:15.482488 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:15.482491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:15.482493 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:15.482576 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:15.482578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.483634 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.483635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:15.484845 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:15.484848 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.485771 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.485772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:15.485861 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:15.485876 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 16:36:15.485955 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 16:36:15.486011 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":85,"threads_started":1}
I20260812 16:36:15.486109 9698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.481815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41094 (local address 127.19.238.66:44451)
0812 16:36:15.482310 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:15.482314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:15.482325 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:36:15.482376 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:15.482378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:15.482391 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:15.482482 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:15.482647 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.483554 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.484942 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.485680 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.485755 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:15.485855 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:15.485982 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 16:36:15.486021 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":118,"threads_started":1}
I20260812 16:36:15.486871 9699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.481884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47522 (local address 127.19.238.67:40735)
0812 16:36:15.482617 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:15.482621 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:15.482631 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:36:15.482669 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:15.482671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:15.482685 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:15.482754 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:15.482928 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.484081 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.485484 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.486086 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.486150 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:15.486396 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:15.486697 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 16:36:15.486768 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":91,"threads_started":1}
I20260812 16:36:15.486915 9701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.482063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49474 (local address 127.19.238.65:41167)
0812 16:36:15.482299 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:15.482304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:15.482989 (+ 685us) server_negotiation.cc:407] Connection header received
0812 16:36:15.483036 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:15.483038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:15.483053 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:15.483131 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:15.483240 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.484288 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.485494 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.486203 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.486273 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:15.486360 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:15.486785 (+ 425us) server_negotiation.cc:299] Negotiation successful
0812 16:36:15.486827 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":118,"threads_started":1}
I20260812 16:36:15.487149 9458 tablet_service.cc:1467] Processing CreateTablet for tablet 74332393b5784c8988eca47dd1dcf35e (DEFAULT_TABLE table=test-table [id=abfe40d7406e4aa48a5189d65917feab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:36:15.487486 9458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74332393b5784c8988eca47dd1dcf35e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:15.486650 9700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.481931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:41167 (local address 127.0.0.1:49474)
0812 16:36:15.482831 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:15.482857 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:15.482924 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:15.483142 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:15.483143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:15.483145 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:15.483179 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:15.483180 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.484381 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.484382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:15.485390 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:15.485393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.486393 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.486394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:15.486440 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:15.486443 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:36:15.486513 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 16:36:15.486556 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"mutex_wait_us":34,"thread_start_us":75,"threads_started":1}
I20260812 16:36:15.490360 9697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.481699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:40735 (local address 127.0.0.1:47522)
0812 16:36:15.482069 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:15.482108 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:15.482166 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:15.482780 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:15.482780 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:15.482782 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:15.482838 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:15.482839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.484418 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.484419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:15.485381 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:15.485384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.486431 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.486432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:15.486495 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:15.486500 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:36:15.486554 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 16:36:15.486597 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":13,"thread_start_us":96,"threads_started":1}
I20260812 16:36:15.490355 9702 tablet_bootstrap.cc:492] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116: Bootstrap starting.
I20260812 16:36:15.491086 9702 tablet_bootstrap.cc:654] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:15.491123 9303 tablet_service.cc:1467] Processing CreateTablet for tablet 74332393b5784c8988eca47dd1dcf35e (DEFAULT_TABLE table=test-table [id=abfe40d7406e4aa48a5189d65917feab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:36:15.491119 9604 tablet_service.cc:1467] Processing CreateTablet for tablet 74332393b5784c8988eca47dd1dcf35e (DEFAULT_TABLE table=test-table [id=abfe40d7406e4aa48a5189d65917feab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:36:15.491463 9702 log.cc:824] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:15.491495 9604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74332393b5784c8988eca47dd1dcf35e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:15.491495 9303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74332393b5784c8988eca47dd1dcf35e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:15.492293 9702 tablet_bootstrap.cc:492] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116: No bootstrap required, opened a new log
I20260812 16:36:15.492383 9702 ts_tablet_manager.cc:1397] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:36:15.494577 9702 raft_consensus.cc:348] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c00ec10a4f2472980667ae5f5417116" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 44451 } } peers { permanent_uuid: "66e01960704542a5abbde7e9b7f8eac9" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 40735 } } peers { permanent_uuid: "cea417aa588d43eda5f718342f43424f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41167 } }
I20260812 16:36:15.494748 9702 raft_consensus.cc:374] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:15.494793 9702 raft_consensus.cc:729] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c00ec10a4f2472980667ae5f5417116, State: Initialized, Role: FOLLOWER
I20260812 16:36:15.494941 9702 consensus_queue.cc:260] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c00ec10a4f2472980667ae5f5417116" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 44451 } } peers { permanent_uuid: "66e01960704542a5abbde7e9b7f8eac9" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 40735 } } peers { permanent_uuid: "cea417aa588d43eda5f718342f43424f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41167 } }
I20260812 16:36:15.495091 9705 tablet_bootstrap.cc:492] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9: Bootstrap starting.
I20260812 16:36:15.495091 9704 tablet_bootstrap.cc:492] T 74332393b5784c8988eca47dd1dcf35e P cea417aa588d43eda5f718342f43424f: Bootstrap starting.
I20260812 16:36:15.495267 9702 ts_tablet_manager.cc:1428] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:36:15.495401 9531 heartbeater.cc:503] Master 127.19.238.126:33149 was elected leader, sending a full tablet report...
I20260812 16:36:15.495816 9705 tablet_bootstrap.cc:654] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:15.495890 9704 tablet_bootstrap.cc:654] T 74332393b5784c8988eca47dd1dcf35e P cea417aa588d43eda5f718342f43424f: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:15.496194 9705 log.cc:824] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:15.496205 9704 log.cc:824] T 74332393b5784c8988eca47dd1dcf35e P cea417aa588d43eda5f718342f43424f: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:15.496950 9704 tablet_bootstrap.cc:492] T 74332393b5784c8988eca47dd1dcf35e P cea417aa588d43eda5f718342f43424f: No bootstrap required, opened a new log
I20260812 16:36:15.497036 9704 ts_tablet_manager.cc:1397] T 74332393b5784c8988eca47dd1dcf35e P cea417aa588d43eda5f718342f43424f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:36:15.497179 9705 tablet_bootstrap.cc:492] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9: No bootstrap required, opened a new log
I20260812 16:36:15.497233 9705 ts_tablet_manager.cc:1397] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 16:36:15.499506 9705 raft_consensus.cc:348] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c00ec10a4f2472980667ae5f5417116" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 44451 } } peers { permanent_uuid: "66e01960704542a5abbde7e9b7f8eac9" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 40735 } } peers { permanent_uuid: "cea417aa588d43eda5f718342f43424f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41167 } }
I20260812 16:36:15.499637 9705 raft_consensus.cc:374] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:15.499658 9705 raft_consensus.cc:729] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e01960704542a5abbde7e9b7f8eac9, State: Initialized, Role: FOLLOWER
I20260812 16:36:15.499780 9705 consensus_queue.cc:260] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c00ec10a4f2472980667ae5f5417116" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 44451 } } peers { permanent_uuid: "66e01960704542a5abbde7e9b7f8eac9" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 40735 } } peers { permanent_uuid: "cea417aa588d43eda5f718342f43424f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41167 } }
I20260812 16:36:15.500092 9705 ts_tablet_manager.cc:1428] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 16:36:15.500149 9688 heartbeater.cc:503] Master 127.19.238.126:33149 was elected leader, sending a full tablet report...
I20260812 16:36:15.500417 9704 raft_consensus.cc:348] T 74332393b5784c8988eca47dd1dcf35e P cea417aa588d43eda5f718342f43424f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c00ec10a4f2472980667ae5f5417116" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 44451 } } peers { permanent_uuid: "66e01960704542a5abbde7e9b7f8eac9" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 40735 } } peers { permanent_uuid: "cea417aa588d43eda5f718342f43424f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41167 } }
I20260812 16:36:15.500558 9704 raft_consensus.cc:374] T 74332393b5784c8988eca47dd1dcf35e P cea417aa588d43eda5f718342f43424f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:15.500581 9704 raft_consensus.cc:729] T 74332393b5784c8988eca47dd1dcf35e P cea417aa588d43eda5f718342f43424f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea417aa588d43eda5f718342f43424f, State: Initialized, Role: FOLLOWER
I20260812 16:36:15.500712 9704 consensus_queue.cc:260] T 74332393b5784c8988eca47dd1dcf35e P cea417aa588d43eda5f718342f43424f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c00ec10a4f2472980667ae5f5417116" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 44451 } } peers { permanent_uuid: "66e01960704542a5abbde7e9b7f8eac9" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 40735 } } peers { permanent_uuid: "cea417aa588d43eda5f718342f43424f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41167 } }
I20260812 16:36:15.500955 9704 ts_tablet_manager.cc:1428] T 74332393b5784c8988eca47dd1dcf35e P cea417aa588d43eda5f718342f43424f: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 16:36:15.501037 9378 heartbeater.cc:503] Master 127.19.238.126:33149 was elected leader, sending a full tablet report...
W20260812 16:36:15.565737 9532 tablet.cc:2369] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:36:15.622653 9706 raft_consensus.cc:482] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:36:15.622821 9706 raft_consensus.cc:504] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c00ec10a4f2472980667ae5f5417116" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 44451 } } peers { permanent_uuid: "66e01960704542a5abbde7e9b7f8eac9" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 40735 } } peers { permanent_uuid: "cea417aa588d43eda5f718342f43424f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41167 } }
I20260812 16:36:15.623225 9706 leader_election.cc:288] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66e01960704542a5abbde7e9b7f8eac9 (127.19.238.67:40735), cea417aa588d43eda5f718342f43424f (127.19.238.65:41167)
I20260812 16:36:15.626428 9533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.623515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:40735 (local address 127.19.238.66:35927)
0812 16:36:15.623577 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:15.623591 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:15.623643 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:15.623898 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:15.623899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:15.623901 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:15.623991 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:15.623994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.624793 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.624794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:15.625570 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:15.625572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.626137 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.626138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:15.626214 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:15.626218 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:36:15.626278 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 16:36:15.626320 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 16:36:15.626428 9699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.623611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:35927 (local address 127.19.238.67:40735)
0812 16:36:15.623677 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:15.623679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:15.623694 (+ 15us) server_negotiation.cc:407] Connection header received
0812 16:36:15.623731 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:15.623732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:15.623746 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:15.623876 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:15.624092 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.624704 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.625665 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.626058 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.626115 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:15.626122 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:15.626287 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 16:36:15.626334 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:36:15.626549 9701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.623792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:54067 (local address 127.19.238.65:41167)
0812 16:36:15.623857 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:15.623859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:15.623980 (+ 121us) server_negotiation.cc:407] Connection header received
0812 16:36:15.624031 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:15.624032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:15.624045 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:15.624101 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:15.624257 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.624782 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.625747 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.626153 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.626222 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:15.626230 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:15.626426 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 16:36:15.626474 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:36:15.626554 9712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.623657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:41167 (local address 127.19.238.66:54067)
0812 16:36:15.623896 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:15.623908 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:15.623967 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:15.624114 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:15.624115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:15.624117 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:15.624184 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:15.624186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.624870 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.624871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:15.625601 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:15.625602 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.626246 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.626247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:15.626358 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:15.626362 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:36:15.626418 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 16:36:15.626461 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":124,"threads_started":1}
I20260812 16:36:15.626880 9325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74332393b5784c8988eca47dd1dcf35e" candidate_uuid: "1c00ec10a4f2472980667ae5f5417116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea417aa588d43eda5f718342f43424f" is_pre_election: true
I20260812 16:36:15.626879 9630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74332393b5784c8988eca47dd1dcf35e" candidate_uuid: "1c00ec10a4f2472980667ae5f5417116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e01960704542a5abbde7e9b7f8eac9" is_pre_election: true
I20260812 16:36:15.627046 9630 raft_consensus.cc:2454] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c00ec10a4f2472980667ae5f5417116 in term 0.
I20260812 16:36:15.627046 9325 raft_consensus.cc:2454] T 74332393b5784c8988eca47dd1dcf35e P cea417aa588d43eda5f718342f43424f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c00ec10a4f2472980667ae5f5417116 in term 0.
I20260812 16:36:15.627292 9401 leader_election.cc:302] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c00ec10a4f2472980667ae5f5417116, 66e01960704542a5abbde7e9b7f8eac9; no voters:
I20260812 16:36:15.627461 9706 raft_consensus.cc:2781] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:36:15.627542 9706 raft_consensus.cc:482] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:36:15.627574 9706 raft_consensus.cc:3037] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:15.628328 9706 raft_consensus.cc:504] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c00ec10a4f2472980667ae5f5417116" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 44451 } } peers { permanent_uuid: "66e01960704542a5abbde7e9b7f8eac9" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 40735 } } peers { permanent_uuid: "cea417aa588d43eda5f718342f43424f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41167 } }
I20260812 16:36:15.628469 9706 leader_election.cc:288] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [CANDIDATE]: Term 1 election: Requested vote from peers 66e01960704542a5abbde7e9b7f8eac9 (127.19.238.67:40735), cea417aa588d43eda5f718342f43424f (127.19.238.65:41167)
I20260812 16:36:15.628724 9325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74332393b5784c8988eca47dd1dcf35e" candidate_uuid: "1c00ec10a4f2472980667ae5f5417116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea417aa588d43eda5f718342f43424f"
I20260812 16:36:15.628722 9630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74332393b5784c8988eca47dd1dcf35e" candidate_uuid: "1c00ec10a4f2472980667ae5f5417116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e01960704542a5abbde7e9b7f8eac9"
I20260812 16:36:15.628813 9325 raft_consensus.cc:3037] T 74332393b5784c8988eca47dd1dcf35e P cea417aa588d43eda5f718342f43424f [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:15.628829 9630 raft_consensus.cc:3037] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:15.629765 9630 raft_consensus.cc:2454] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c00ec10a4f2472980667ae5f5417116 in term 1.
I20260812 16:36:15.629823 9325 raft_consensus.cc:2454] T 74332393b5784c8988eca47dd1dcf35e P cea417aa588d43eda5f718342f43424f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c00ec10a4f2472980667ae5f5417116 in term 1.
I20260812 16:36:15.629954 9401 leader_election.cc:302] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c00ec10a4f2472980667ae5f5417116, cea417aa588d43eda5f718342f43424f; no voters:
I20260812 16:36:15.630057 9706 raft_consensus.cc:2781] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:15.630234 9706 raft_consensus.cc:686] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [term 1 LEADER]: Becoming Leader. State: Replica: 1c00ec10a4f2472980667ae5f5417116, State: Running, Role: LEADER
I20260812 16:36:15.630370 9706 consensus_queue.cc:237] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c00ec10a4f2472980667ae5f5417116" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 44451 } } peers { permanent_uuid: "66e01960704542a5abbde7e9b7f8eac9" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 40735 } } peers { permanent_uuid: "cea417aa588d43eda5f718342f43424f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41167 } }
I20260812 16:36:15.631251 9164 catalog_manager.cc:5495] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c00ec10a4f2472980667ae5f5417116 (127.19.238.66). New cstate: current_term: 1 leader_uuid: "1c00ec10a4f2472980667ae5f5417116" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c00ec10a4f2472980667ae5f5417116" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 44451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66e01960704542a5abbde7e9b7f8eac9" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 40735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea417aa588d43eda5f718342f43424f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41167 } health_report { overall_health: UNKNOWN } } }
I20260812 16:36:15.668421 9698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.665870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41102 (local address 127.19.238.66:44451)
0812 16:36:15.665969 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:15.665971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:15.665979 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:36:15.666017 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:15.666018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:15.666038 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:15.666122 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:36:15.666272 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.666796 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.667565 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:15.667767 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:15.667869 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:15.668032 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:36:15.668135 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:36:15.668249 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:15.668280 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 16:36:15.668291 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 16:36:15.670702 9630 raft_consensus.cc:1261] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9 [term 1 FOLLOWER]: Refusing update from remote peer 1c00ec10a4f2472980667ae5f5417116: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:36:15.670702 9325 raft_consensus.cc:1261] T 74332393b5784c8988eca47dd1dcf35e P cea417aa588d43eda5f718342f43424f [term 1 FOLLOWER]: Refusing update from remote peer 1c00ec10a4f2472980667ae5f5417116: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:36:15.671126 9713 consensus_queue.cc:1035] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e01960704542a5abbde7e9b7f8eac9" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 40735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:36:15.671276 9706 consensus_queue.cc:1035] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea417aa588d43eda5f718342f43424f" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 41167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:36:15.676199 9718 mvcc.cc:205] Tried to move back new op lower bound from 7317719349943549952 to 7317719349782466560. Current Snapshot: MvccSnapshot[applied={T|T < 7317719349943549952 or (T in {7317719349943549952})}]
I20260812 16:36:15.676492 9164 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:36:15.676607 9164 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:36:15.677430 9164 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e63ac311f1174aa0b3f8adfe4a286a33: Sending DeleteTablet for 3 replicas of tablet 74332393b5784c8988eca47dd1dcf35e
I20260812 16:36:15.677786 9604 tablet_service.cc:1514] Processing DeleteTablet for tablet 74332393b5784c8988eca47dd1dcf35e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:36:15 UTC) from {username='slave'} at 127.0.0.1:47522
I20260812 16:36:15.677837 9458 tablet_service.cc:1514] Processing DeleteTablet for tablet 74332393b5784c8988eca47dd1dcf35e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:36:15 UTC) from {username='slave'} at 127.0.0.1:41094
I20260812 16:36:15.677861 9303 tablet_service.cc:1514] Processing DeleteTablet for tablet 74332393b5784c8988eca47dd1dcf35e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:36:15 UTC) from {username='slave'} at 127.0.0.1:49474
I20260812 16:36:15.678085 9729 tablet_replica.cc:330] T 74332393b5784c8988eca47dd1dcf35e P cea417aa588d43eda5f718342f43424f: stopping tablet replica
I20260812 16:36:15.678180 9729 raft_consensus.cc:2229] T 74332393b5784c8988eca47dd1dcf35e P cea417aa588d43eda5f718342f43424f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:36:15.678289 9729 raft_consensus.cc:2258] T 74332393b5784c8988eca47dd1dcf35e P cea417aa588d43eda5f718342f43424f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:36:15.678328 9727 tablet_replica.cc:330] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9: stopping tablet replica
I20260812 16:36:15.678426 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 9232
I20260812 16:36:15.678548 9728 tablet_replica.cc:330] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116: stopping tablet replica
I20260812 16:36:15.678643 9728 raft_consensus.cc:2229] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:36:15.678728 9728 raft_consensus.cc:2258] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:36:15.683965 9727 raft_consensus.cc:2229] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:36:15.684130 9727 raft_consensus.cc:2258] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:36:15.684650 9727 ts_tablet_manager.cc:1905] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:36:15.686545 9728 ts_tablet_manager.cc:1905] T 74332393b5784c8988eca47dd1dcf35e P 1c00ec10a4f2472980667ae5f5417116: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:36:15.686592 9727 ts_tablet_manager.cc:1918] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:36:15.686686 9727 log.cc:1196] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/74332393b5784c8988eca47dd1dcf35e
I20260812 16:36:15.686973 9727 ts_tablet_manager.cc:1939] T 74332393b5784c8988eca47dd1dcf35e P 66e01960704542a5abbde7e9b7f8eac9: Deleting consensus metadata
I20260812 16:36:15.687433 9150 catalog_manager.cc:4841] TS 66e01960704542a5abbde7e9b7f8eac9 (127.19.238.67:40735): tablet 74332393b5784c8988eca47dd1dcf35e (table test-table [id=abfe40d7406e4aa48a5189d65917feab]) successfully deleted
W20260812 16:36:15.687999 9150 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.238.65:41167 (error 108)
I20260812 16:36:15.688293 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 9381
I20260812 16:36:15.693536 9697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.693318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:41167 (local address 127.0.0.1:49478)
0812 16:36:15.693417 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:15.693489 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:41167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 16:36:15.693742 9150 catalog_manager.cc:4584] TS cea417aa588d43eda5f718342f43424f (127.19.238.65:41167): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 74332393b5784c8988eca47dd1dcf35e: Network error: Client connection negotiation failed: client connection to 127.19.238.65:41167: connect: Connection refused (error 111)
I20260812 16:36:15.696898 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 9534
I20260812 16:36:15.696966 9697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:15.696825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:44451 (local address 127.0.0.1:41110)
0812 16:36:15.696891 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:15.696928 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.66:44451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 16:36:15.703308 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 9128
2026-08-12T16:36:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (820 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:15 dist-test-slave-mvvj krb5kdc[9733](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:15 dist-test-slave-mvvj krb5kdc[9733](info): set up 2 sockets
Aug 12 16:36:15 dist-test-slave-mvvj krb5kdc[9733](info): commencing operation
krb5kdc: starting...
W20260812 16:36:16.812029 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:16 dist-test-slave-mvvj krb5kdc[9733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:36:16.943570 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:43047
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:44575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:43047
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:17.028855 9750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:17.029124 9750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:17.029165 9750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:17.030380 9750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:17.030449 9750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:17.030475 9750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:17.030507 9750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:17.030537 9750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:17.032188 9750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:44575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:43047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:43047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:17.032473 9750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:17.032725 9750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:17.037932 9755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:17.037966 9756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:17.038036 9750 server_base.cc:1034] running on GCE node
W20260812 16:36:17.037954 9758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:17.038489 9750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:17.038839 9750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:17.040021 9750 hybrid_clock.cc:648] HybridClock initialized: now 1786552577039997 us; error 60 us; skew 500 ppm
Aug 12 16:36:17 dist-test-slave-mvvj krb5kdc[9733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552577, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:17.042723 9750 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:36:17.043129 9750 webserver.cc:460] Webserver started at http://127.19.238.126:38603/ using document root <none> and password file <none>
I20260812 16:36:17.043316 9750 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:17.043365 9750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:17.043454 9750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:17.044585 9750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "893e04f9038344ee8c2b5b5aa2ceb716"
format_stamp: "Formatted at 2026-08-12 16:36:17 on dist-test-slave-mvvj"
server_key: "a9729ec67b81a4afca503be15ff6d511"
server_key_iv: "555f25bea668492cc2371c6f2616574c"
server_key_version: "encryptionkey@0"
I20260812 16:36:17.044919 9750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "893e04f9038344ee8c2b5b5aa2ceb716"
format_stamp: "Formatted at 2026-08-12 16:36:17 on dist-test-slave-mvvj"
server_key: "a9729ec67b81a4afca503be15ff6d511"
server_key_iv: "555f25bea668492cc2371c6f2616574c"
server_key_version: "encryptionkey@0"
I20260812 16:36:17.046343 9750 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:36:17.047154 9767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:17.047349 9750 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:17.047436 9750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "893e04f9038344ee8c2b5b5aa2ceb716"
format_stamp: "Formatted at 2026-08-12 16:36:17 on dist-test-slave-mvvj"
server_key: "a9729ec67b81a4afca503be15ff6d511"
server_key_iv: "555f25bea668492cc2371c6f2616574c"
server_key_version: "encryptionkey@0"
I20260812 16:36:17.047497 9750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:17.062966 9750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:17.065362 9750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:17.065482 9750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:17.069407 9750 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:43047
I20260812 16:36:17.069485 9828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:43047 every 8 connection(s)
I20260812 16:36:17.069850 9750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:17.070803 9831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:17.073273 9831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716: Bootstrap starting.
I20260812 16:36:17.073875 9831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:17.074177 9831 log.cc:824] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:17.074895 9831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716: No bootstrap required, opened a new log
I20260812 16:36:17.076632 9831 raft_consensus.cc:348] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "893e04f9038344ee8c2b5b5aa2ceb716" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 43047 } }
I20260812 16:36:17.076813 9831 raft_consensus.cc:374] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:17.076865 9831 raft_consensus.cc:729] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 893e04f9038344ee8c2b5b5aa2ceb716, State: Initialized, Role: FOLLOWER
I20260812 16:36:17.077068 9831 consensus_queue.cc:260] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "893e04f9038344ee8c2b5b5aa2ceb716" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 43047 } }
I20260812 16:36:17.077173 9831 raft_consensus.cc:388] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:17.077220 9831 raft_consensus.cc:482] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:17.077270 9831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:17.077831 9831 raft_consensus.cc:504] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "893e04f9038344ee8c2b5b5aa2ceb716" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 43047 } }
I20260812 16:36:17.077957 9831 leader_election.cc:302] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 893e04f9038344ee8c2b5b5aa2ceb716; no voters:
I20260812 16:36:17.078158 9831 leader_election.cc:288] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:17.078209 9835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:17.078428 9835 raft_consensus.cc:686] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716 [term 1 LEADER]: Becoming Leader. State: Replica: 893e04f9038344ee8c2b5b5aa2ceb716, State: Running, Role: LEADER
I20260812 16:36:17.078439 9831 sys_catalog.cc:564] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:17.078531 9835 consensus_queue.cc:237] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "893e04f9038344ee8c2b5b5aa2ceb716" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 43047 } }
I20260812 16:36:17.078920 9837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 893e04f9038344ee8c2b5b5aa2ceb716. Latest consensus state: current_term: 1 leader_uuid: "893e04f9038344ee8c2b5b5aa2ceb716" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "893e04f9038344ee8c2b5b5aa2ceb716" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 43047 } } }
I20260812 16:36:17.079017 9837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:17.079056 9836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "893e04f9038344ee8c2b5b5aa2ceb716" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "893e04f9038344ee8c2b5b5aa2ceb716" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 43047 } } }
I20260812 16:36:17.079129 9836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:17.079265 9842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:17.079584 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 9750
I20260812 16:36:17.079706 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:17.079864 9842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:17.079953 20409 external_mini_cluster.cc:1428] Setting key 8358b4ec51ab8e85e07a11cb75dcff3b
I20260812 16:36:17.081725 9842 catalog_manager.cc:1347] Generated new cluster ID: ead66f0e328549208583ac55ce521244
I20260812 16:36:17.081781 9842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:36:17 dist-test-slave-mvvj krb5kdc[9733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:36:17.092052 9858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.080429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53956 (local address 127.19.238.126:43047)
0812 16:36:17.080798 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:17.080806 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:17.080830 (+ 24us) server_negotiation.cc:407] Connection header received
0812 16:36:17.080964 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:17.080971 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:17.081030 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:17.081183 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:17.081946 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.083371 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.084450 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.085096 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.085230 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.088188 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:17.088192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:17.088193 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:17.088205 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:17.090253 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:17.090688 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:17.090691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:17.090692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:17.090795 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:17.091023 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:17.091023 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:17.091024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:17.091333 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:17.091513 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:17.091628 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 16:36:17.091764 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":78,"threads_started":1}
I20260812 16:36:17.101816 9842 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:17.102607 9842 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:17.115695 9842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716: Generated new TSK 0
I20260812 16:36:17.115971 9842 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.238.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
Entry for principal kudu/127.19.238.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab.
I20260812 16:36:17.150862 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:43047
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:44575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:17.239116 9862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:17.239373 9862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:17.239425 9862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:17.240783 9862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:17.240861 9862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:17.240909 9862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:36:17.242475 9862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:44575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.65.keytab
--principal=kudu/127.19.238.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:43047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:17.242765 9862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:17.243018 9862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:17.248342 9871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:17.248318 9869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:17.248318 9868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:17.248734 9862 server_base.cc:1034] running on GCE node
I20260812 16:36:17.248926 9862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:17.249148 9862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:17.250249 9862 hybrid_clock.cc:648] HybridClock initialized: now 1786552577250206 us; error 57 us; skew 500 ppm
Aug 12 16:36:17 dist-test-slave-mvvj krb5kdc[9733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552577, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:17.252995 9862 init.cc:377] Logged in from keytab as kudu/127.19.238.65@KRBTEST.COM (short username kudu)
I20260812 16:36:17.253506 9862 webserver.cc:460] Webserver started at http://127.19.238.65:45169/ using document root <none> and password file <none>
I20260812 16:36:17.253748 9862 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:17.253815 9862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:17.253973 9862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:17.255045 9862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "dcbf1ee01bc9412bba5a59422097fe02"
format_stamp: "Formatted at 2026-08-12 16:36:17 on dist-test-slave-mvvj"
server_key: "dce4158f4da06461d7a6b6b6d17424fa"
server_key_iv: "7392da5801e7965fe8faba26051dc98c"
server_key_version: "encryptionkey@0"
I20260812 16:36:17.255427 9862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "dcbf1ee01bc9412bba5a59422097fe02"
format_stamp: "Formatted at 2026-08-12 16:36:17 on dist-test-slave-mvvj"
server_key: "dce4158f4da06461d7a6b6b6d17424fa"
server_key_iv: "7392da5801e7965fe8faba26051dc98c"
server_key_version: "encryptionkey@0"
I20260812 16:36:17.256948 9862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:36:17.257828 9879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:17.258054 9862 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:36:17.258160 9862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "dcbf1ee01bc9412bba5a59422097fe02"
format_stamp: "Formatted at 2026-08-12 16:36:17 on dist-test-slave-mvvj"
server_key: "dce4158f4da06461d7a6b6b6d17424fa"
server_key_iv: "7392da5801e7965fe8faba26051dc98c"
server_key_version: "encryptionkey@0"
I20260812 16:36:17.258241 9862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:17.269461 9862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:17.271773 9862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:17.272007 9862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:17.272269 9862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:36:17.272615 9862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:36:17.272679 9862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:17.272717 9862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:36:17.272764 9862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:17.279812 9862 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:37189
I20260812 16:36:17.279933 10011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:37189 every 8 connection(s)
I20260812 16:36:17.280288 9862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
Aug 12 16:36:17 dist-test-slave-mvvj krb5kdc[9733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552577, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.65@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:36:17.288822 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 9862
I20260812 16:36:17.288929 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:36:17.289187 20409 external_mini_cluster.cc:1428] Setting key f6ce3fa5678a4e4bfd8c9c9cfb5e0ed0
I20260812 16:36:17.290992 9858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.281086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:43855 (local address 127.19.238.126:43047)
0812 16:36:17.281157 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:17.281159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:17.281471 (+ 312us) server_negotiation.cc:407] Connection header received
0812 16:36:17.281966 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:17.281967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:17.281980 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:17.282044 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:17.283027 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.283562 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.284681 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.284895 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.284974 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.287462 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:17.287463 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:17.287464 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:17.287473 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:17.289112 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:17.289752 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:17.289753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:17.289754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:17.289852 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:17.290201 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:17.290202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:17.290203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:17.290366 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:17.290453 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:17.290754 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 16:36:17.290866 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 16:36:17.291024 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.281071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:43047 (local address 127.19.238.65:43855)
0812 16:36:17.281250 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:17.281297 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:17.281876 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:17.282096 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:17.282100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:17.282707 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:17.282912 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:17.282917 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.283672 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.283673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:17.284581 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:17.284584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.285000 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.285001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:17.285109 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.285791 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:17.285800 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:17.287341 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:17.289196 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:17.289204 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:17.289206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:17.289621 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:17.289960 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:17.289961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:17.289961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:17.290125 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:17.290480 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:17.290486 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:17.290616 (+ 130us) client_negotiation.cc:769] Sending connection context
0812 16:36:17.290705 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 16:36:17.290837 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":96,"threads_started":1}
I20260812 16:36:17.291599 10012 heartbeater.cc:347] Connected to a master server at 127.19.238.126:43047
I20260812 16:36:17.291723 10012 heartbeater.cc:464] Registering TS with master...
I20260812 16:36:17.291951 10012 heartbeater.cc:511] Master 127.19.238.126:43047 requested a full tablet report, sending...
I20260812 16:36:17.292541 9790 ts_manager.cc:194] Registered new tserver with Master: dcbf1ee01bc9412bba5a59422097fe02 (127.19.238.65:37189)
I20260812 16:36:17.293172 9790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.65@KRBTEST.COM'} at 127.19.238.65:43855
WARNING: no policy specified for kudu/127.19.238.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
Entry for principal kudu/127.19.238.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab.
I20260812 16:36:17.319142 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:43047
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:44575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:17.404393 10017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:17.404685 10017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:17.404713 10017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:17.405834 10017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:17.405870 10017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:17.405896 10017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:36:17.407298 10017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:44575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.66.keytab
--principal=kudu/127.19.238.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:43047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:17.407495 10017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:17.407678 10017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:17.413203 10025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:17.413264 10017 server_base.cc:1034] running on GCE node
W20260812 16:36:17.413213 10023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:17.413230 10022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:17.413707 10017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:17.413931 10017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:17.415074 10017 hybrid_clock.cc:648] HybridClock initialized: now 1786552577415054 us; error 41 us; skew 500 ppm
Aug 12 16:36:17 dist-test-slave-mvvj krb5kdc[9733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552577, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:17.417918 10017 init.cc:377] Logged in from keytab as kudu/127.19.238.66@KRBTEST.COM (short username kudu)
I20260812 16:36:17.418462 10017 webserver.cc:460] Webserver started at http://127.19.238.66:41737/ using document root <none> and password file <none>
I20260812 16:36:17.418753 10017 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:17.418820 10017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:17.418969 10017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:17.420166 10017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "23f2fc19c97a446c981d92cf3ffe442e"
format_stamp: "Formatted at 2026-08-12 16:36:17 on dist-test-slave-mvvj"
server_key: "39e49723057b1deb7e14ae91c8189373"
server_key_iv: "7aefd0f22fe6077682c6d2eef666e8d2"
server_key_version: "encryptionkey@0"
I20260812 16:36:17.420488 10017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "23f2fc19c97a446c981d92cf3ffe442e"
format_stamp: "Formatted at 2026-08-12 16:36:17 on dist-test-slave-mvvj"
server_key: "39e49723057b1deb7e14ae91c8189373"
server_key_iv: "7aefd0f22fe6077682c6d2eef666e8d2"
server_key_version: "encryptionkey@0"
I20260812 16:36:17.421885 10017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:36:17.422732 10032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:17.422916 10017 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:17.423017 10017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "23f2fc19c97a446c981d92cf3ffe442e"
format_stamp: "Formatted at 2026-08-12 16:36:17 on dist-test-slave-mvvj"
server_key: "39e49723057b1deb7e14ae91c8189373"
server_key_iv: "7aefd0f22fe6077682c6d2eef666e8d2"
server_key_version: "encryptionkey@0"
I20260812 16:36:17.423086 10017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:17.431095 10017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:17.433408 10017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:17.433518 10017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:17.433701 10017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:36:17.434029 10017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:36:17.434065 10017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:17.434087 10017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:36:17.434135 10017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:17.440900 10017 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:39795
I20260812 16:36:17.441305 10017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:36:17.441121 10161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:39795 every 8 connection(s)
I20260812 16:36:17.445272 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 10017
I20260812 16:36:17.445389 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:36:17.445645 20409 external_mini_cluster.cc:1428] Setting key 13cebd092f5137c1543e84bbe232b959
Aug 12 16:36:17 dist-test-slave-mvvj krb5kdc[9733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552577, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.66@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.238.67@KRBTEST.COM; defaulting to no policy
I20260812 16:36:17.457299 9858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.445874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:47423 (local address 127.19.238.126:43047)
0812 16:36:17.445968 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:17.445971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:17.446451 (+ 480us) server_negotiation.cc:407] Connection header received
0812 16:36:17.447247 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:17.447249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:17.447261 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:17.447340 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:17.448707 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.449220 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.450737 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.450970 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.451045 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.453648 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:17.453649 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:17.453650 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:17.453657 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:17.455200 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:17.455783 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:17.455785 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:17.455785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:17.455879 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:17.456224 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:17.456226 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:17.456226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:17.456398 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:17.456490 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:17.457043 (+ 553us) server_negotiation.cc:299] Negotiation successful
0812 16:36:17.457159 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 16:36:17.457402 10165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.445881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:43047 (local address 127.19.238.66:47423)
0812 16:36:17.446313 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:17.446365 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:17.447133 (+ 768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:17.447416 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:17.447420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:17.448314 (+ 894us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:17.448621 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:17.448628 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.449325 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.449327 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:17.450600 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:17.450604 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.451078 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.451079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:17.451237 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.451797 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:17.451808 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:17.453521 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:17.455309 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:17.455318 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:17.455320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:17.455684 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:17.455971 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:17.455972 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:17.455972 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:17.456126 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:17.456482 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:17.456637 (+ 155us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:17.456844 (+ 207us) client_negotiation.cc:769] Sending connection context
0812 16:36:17.456975 (+ 131us) client_negotiation.cc:240] Negotiation successful
0812 16:36:17.457164 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":102,"threads_started":1}
I20260812 16:36:17.458029 10162 heartbeater.cc:347] Connected to a master server at 127.19.238.126:43047
I20260812 16:36:17.458153 10162 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.67@KRBTEST.COM" created.
I20260812 16:36:17.458385 10162 heartbeater.cc:511] Master 127.19.238.126:43047 requested a full tablet report, sending...
I20260812 16:36:17.458834 9790 ts_manager.cc:194] Registered new tserver with Master: 23f2fc19c97a446c981d92cf3ffe442e (127.19.238.66:39795)
I20260812 16:36:17.459370 9790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.66@KRBTEST.COM'} at 127.19.238.66:47423
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
Entry for principal kudu/127.19.238.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab.
I20260812 16:36:17.477170 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:43047
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:44575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:17.568846 10168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:17.569111 10168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:17.569157 10168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:17.570457 10168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:17.570535 10168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:17.570587 10168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:36:17.572333 10168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:44575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.67.keytab
--principal=kudu/127.19.238.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:43047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:17.572655 10168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:17.572916 10168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:17.578697 10174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:17.578688 10177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:17.578718 10175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:17.579124 10168 server_base.cc:1034] running on GCE node
I20260812 16:36:17.579342 10168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:17.579630 10168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:17.580798 10168 hybrid_clock.cc:648] HybridClock initialized: now 1786552577580778 us; error 31 us; skew 500 ppm
Aug 12 16:36:17 dist-test-slave-mvvj krb5kdc[9733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552577, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:17.583380 10168 init.cc:377] Logged in from keytab as kudu/127.19.238.67@KRBTEST.COM (short username kudu)
I20260812 16:36:17.583969 10168 webserver.cc:460] Webserver started at http://127.19.238.67:37119/ using document root <none> and password file <none>
I20260812 16:36:17.584172 10168 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:17.584213 10168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:17.584306 10168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:17.585309 10168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "41f862f411754184980ef7f5c3566e97"
format_stamp: "Formatted at 2026-08-12 16:36:17 on dist-test-slave-mvvj"
server_key: "75ad07f89ae34f3f0bccb126b1d3284e"
server_key_iv: "9d61f75498c6c0084295bc4eb62417d1"
server_key_version: "encryptionkey@0"
I20260812 16:36:17.585629 10168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "41f862f411754184980ef7f5c3566e97"
format_stamp: "Formatted at 2026-08-12 16:36:17 on dist-test-slave-mvvj"
server_key: "75ad07f89ae34f3f0bccb126b1d3284e"
server_key_iv: "9d61f75498c6c0084295bc4eb62417d1"
server_key_version: "encryptionkey@0"
I20260812 16:36:17.587007 10168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:36:17.587872 10185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:17.588042 10168 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:36:17.588136 10168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "41f862f411754184980ef7f5c3566e97"
format_stamp: "Formatted at 2026-08-12 16:36:17 on dist-test-slave-mvvj"
server_key: "75ad07f89ae34f3f0bccb126b1d3284e"
server_key_iv: "9d61f75498c6c0084295bc4eb62417d1"
server_key_version: "encryptionkey@0"
I20260812 16:36:17.588196 10168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:17.604233 10168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:17.606194 10168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:17.606307 10168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:17.606505 10168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:36:17.606837 10168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:36:17.606871 10168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:17.606894 10168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:36:17.606942 10168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:17.612864 10168 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:36411
I20260812 16:36:17.612910 10314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:36411 every 8 connection(s)
I20260812 16:36:17.613236 10168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
Aug 12 16:36:17 dist-test-slave-mvvj krb5kdc[9733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552577, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.67@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:36:17.623348 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 10168
I20260812 16:36:17.623548 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:36:17.623754 20409 external_mini_cluster.cc:1428] Setting key 5f872dd2b0c9651521e69b0c9bf90264
I20260812 16:36:17.623931 9858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.614034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:44189 (local address 127.19.238.126:43047)
0812 16:36:17.614137 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:17.614140 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:17.614327 (+ 187us) server_negotiation.cc:407] Connection header received
0812 16:36:17.614960 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:17.614963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:17.614976 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:17.615041 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:17.616172 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.616736 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.617829 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.618064 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.618152 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.620751 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:17.620753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:17.620754 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:17.620762 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:17.622302 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:17.622761 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:17.622762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:17.622763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:17.622830 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:17.623129 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:17.623130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:17.623130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:17.623275 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:17.623357 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:17.623708 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 16:36:17.623796 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 16:36:17.624136 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.613984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:43047 (local address 127.19.238.67:44189)
0812 16:36:17.614179 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:17.614235 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:17.614816 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:17.615118 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:17.615123 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:17.615848 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:17.616060 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:17.616065 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.616846 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.616847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:17.617724 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:17.617727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.618159 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.618161 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:17.618320 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.618879 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:17.618888 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:17.620640 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:17.622421 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:17.622428 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:17.622430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:17.622674 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:17.622908 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:17.622909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:17.622910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:17.623043 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:17.623362 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:17.623369 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:17.623543 (+ 174us) client_negotiation.cc:769] Sending connection context
0812 16:36:17.623668 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 16:36:17.623875 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":96,"threads_started":1}
I20260812 16:36:17.624596 10315 heartbeater.cc:347] Connected to a master server at 127.19.238.126:43047
I20260812 16:36:17.624698 10315 heartbeater.cc:464] Registering TS with master...
I20260812 16:36:17.624907 10315 heartbeater.cc:511] Master 127.19.238.126:43047 requested a full tablet report, sending...
I20260812 16:36:17.625306 9790 ts_manager.cc:194] Registered new tserver with Master: 41f862f411754184980ef7f5c3566e97 (127.19.238.67:36411)
I20260812 16:36:17.625700 9790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.238.67@KRBTEST.COM'} at 127.19.238.67:44189
I20260812 16:36:17.635980 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:36:17.643452 9858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.637380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53964 (local address 127.19.238.126:43047)
0812 16:36:17.637485 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:17.637487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:17.637495 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:36:17.637518 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:17.637519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:17.637528 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:17.637585 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:17.638151 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.638678 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.639364 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.639580 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.639636 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.640828 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:17.640830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:17.640830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:17.640837 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:17.642382 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:17.642728 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:17.642729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:17.642730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:17.642787 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:17.642949 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:17.642949 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:17.642950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:17.643099 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:17.643175 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:17.643276 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 16:36:17.643341 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 16:36:17.645092 9790 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:36:17.646940 9790 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:36:17.654152 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.649453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:39795 (local address 127.0.0.1:42634)
0812 16:36:17.649664 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:17.649707 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:17.649770 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:17.650218 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:17.650219 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:17.650221 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:17.650269 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:17.650271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.651538 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.651539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:17.652965 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:17.652968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.653855 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.653856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:17.653937 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.653955 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 16:36:17.654022 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 16:36:17.654077 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":72,"thread_start_us":69,"threads_started":1}
I20260812 16:36:17.654405 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.650184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33056 (local address 127.19.238.65:37189)
0812 16:36:17.650457 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:17.650462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:17.650579 (+ 117us) server_negotiation.cc:407] Connection header received
0812 16:36:17.650641 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:17.650644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:17.650702 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:17.650785 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:17.650997 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.652097 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.653389 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.653923 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.654000 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.654071 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:17.654239 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 16:36:17.654292 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":98,"threads_started":1}
I20260812 16:36:17.654543 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.649834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42634 (local address 127.19.238.66:39795)
0812 16:36:17.650022 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:17.650027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:17.650039 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:36:17.650084 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:17.650086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:17.650102 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:17.650219 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:17.650337 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.651423 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.653075 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.653766 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.653831 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.653873 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:17.654387 (+ 514us) server_negotiation.cc:299] Negotiation successful
0812 16:36:17.654445 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":118,"threads_started":1}
I20260812 16:36:17.654767 10330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.649677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:37189 (local address 127.0.0.1:33056)
0812 16:36:17.650506 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:17.650516 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:17.650575 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:17.650873 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:17.650874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:17.650876 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:17.650917 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:17.650919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.652204 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.652205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:17.653289 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:17.653294 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.654060 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.654062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:17.654138 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.654143 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:36:17.654208 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 16:36:17.654614 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"thread_start_us":84,"threads_started":1}
I20260812 16:36:17.655133 10332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.649487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:36411 (local address 127.0.0.1:49092)
0812 16:36:17.649830 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:17.649842 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:17.649913 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:17.650138 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:17.650142 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:17.650144 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:17.650214 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:17.650216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.651712 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.651713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:17.652965 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:17.652968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.654910 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.654911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:17.654972 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.654976 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:36:17.655028 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 16:36:17.655066 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":5,"thread_start_us":100,"threads_started":1}
I20260812 16:36:17.655370 10333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.649683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49092 (local address 127.19.238.67:36411)
0812 16:36:17.649926 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:17.649931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:17.649943 (+ 12us) server_negotiation.cc:407] Connection header received
0812 16:36:17.649990 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:17.649992 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:17.650008 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:17.650112 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:17.650311 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.651594 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.653910 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.654819 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.654912 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.654974 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:17.655242 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 16:36:17.655286 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":119,"threads_started":1}
I20260812 16:36:17.655365 10084 tablet_service.cc:1467] Processing CreateTablet for tablet 4b2e89d90762472f8c8244398adbbc35 (DEFAULT_TABLE table=test-table [id=65bb2065acdb484b9b7ef72d5cfcbe3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:36:17.655707 10084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b2e89d90762472f8c8244398adbbc35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:17.655992 10241 tablet_service.cc:1467] Processing CreateTablet for tablet 4b2e89d90762472f8c8244398adbbc35 (DEFAULT_TABLE table=test-table [id=65bb2065acdb484b9b7ef72d5cfcbe3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:36:17.656338 10241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b2e89d90762472f8c8244398adbbc35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:17.656463 9932 tablet_service.cc:1467] Processing CreateTablet for tablet 4b2e89d90762472f8c8244398adbbc35 (DEFAULT_TABLE table=test-table [id=65bb2065acdb484b9b7ef72d5cfcbe3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:36:17.656792 9932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b2e89d90762472f8c8244398adbbc35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:17.659171 10337 tablet_bootstrap.cc:492] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97: Bootstrap starting.
I20260812 16:36:17.659379 10338 tablet_bootstrap.cc:492] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e: Bootstrap starting.
I20260812 16:36:17.659926 10337 tablet_bootstrap.cc:654] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:17.660087 10338 tablet_bootstrap.cc:654] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:17.660238 10339 tablet_bootstrap.cc:492] T 4b2e89d90762472f8c8244398adbbc35 P dcbf1ee01bc9412bba5a59422097fe02: Bootstrap starting.
I20260812 16:36:17.660555 10337 log.cc:824] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:17.660697 10338 log.cc:824] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:17.660980 10339 tablet_bootstrap.cc:654] T 4b2e89d90762472f8c8244398adbbc35 P dcbf1ee01bc9412bba5a59422097fe02: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:17.661441 10337 tablet_bootstrap.cc:492] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97: No bootstrap required, opened a new log
I20260812 16:36:17.661545 10337 ts_tablet_manager.cc:1397] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:36:17.661571 10339 log.cc:824] T 4b2e89d90762472f8c8244398adbbc35 P dcbf1ee01bc9412bba5a59422097fe02: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:17.661595 10338 tablet_bootstrap.cc:492] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e: No bootstrap required, opened a new log
I20260812 16:36:17.661659 10338 ts_tablet_manager.cc:1397] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:36:17.662331 10339 tablet_bootstrap.cc:492] T 4b2e89d90762472f8c8244398adbbc35 P dcbf1ee01bc9412bba5a59422097fe02: No bootstrap required, opened a new log
I20260812 16:36:17.662406 10339 ts_tablet_manager.cc:1397] T 4b2e89d90762472f8c8244398adbbc35 P dcbf1ee01bc9412bba5a59422097fe02: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:36:17.663245 10338 raft_consensus.cc:348] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f862f411754184980ef7f5c3566e97" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 36411 } } peers { permanent_uuid: "23f2fc19c97a446c981d92cf3ffe442e" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 39795 } } peers { permanent_uuid: "dcbf1ee01bc9412bba5a59422097fe02" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 37189 } }
I20260812 16:36:17.663408 10338 raft_consensus.cc:374] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:17.663452 10338 raft_consensus.cc:729] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23f2fc19c97a446c981d92cf3ffe442e, State: Initialized, Role: FOLLOWER
I20260812 16:36:17.663563 10338 consensus_queue.cc:260] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f862f411754184980ef7f5c3566e97" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 36411 } } peers { permanent_uuid: "23f2fc19c97a446c981d92cf3ffe442e" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 39795 } } peers { permanent_uuid: "dcbf1ee01bc9412bba5a59422097fe02" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 37189 } }
I20260812 16:36:17.663544 10337 raft_consensus.cc:348] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f862f411754184980ef7f5c3566e97" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 36411 } } peers { permanent_uuid: "23f2fc19c97a446c981d92cf3ffe442e" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 39795 } } peers { permanent_uuid: "dcbf1ee01bc9412bba5a59422097fe02" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 37189 } }
I20260812 16:36:17.663718 10337 raft_consensus.cc:374] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:17.663761 10337 raft_consensus.cc:729] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41f862f411754184980ef7f5c3566e97, State: Initialized, Role: FOLLOWER
I20260812 16:36:17.663895 10338 ts_tablet_manager.cc:1428] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 16:36:17.664054 10162 heartbeater.cc:503] Master 127.19.238.126:43047 was elected leader, sending a full tablet report...
I20260812 16:36:17.664064 10337 consensus_queue.cc:260] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f862f411754184980ef7f5c3566e97" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 36411 } } peers { permanent_uuid: "23f2fc19c97a446c981d92cf3ffe442e" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 39795 } } peers { permanent_uuid: "dcbf1ee01bc9412bba5a59422097fe02" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 37189 } }
I20260812 16:36:17.664397 10315 heartbeater.cc:503] Master 127.19.238.126:43047 was elected leader, sending a full tablet report...
I20260812 16:36:17.664577 10339 raft_consensus.cc:348] T 4b2e89d90762472f8c8244398adbbc35 P dcbf1ee01bc9412bba5a59422097fe02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f862f411754184980ef7f5c3566e97" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 36411 } } peers { permanent_uuid: "23f2fc19c97a446c981d92cf3ffe442e" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 39795 } } peers { permanent_uuid: "dcbf1ee01bc9412bba5a59422097fe02" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 37189 } }
I20260812 16:36:17.664757 10339 raft_consensus.cc:374] T 4b2e89d90762472f8c8244398adbbc35 P dcbf1ee01bc9412bba5a59422097fe02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:17.664817 10339 raft_consensus.cc:729] T 4b2e89d90762472f8c8244398adbbc35 P dcbf1ee01bc9412bba5a59422097fe02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcbf1ee01bc9412bba5a59422097fe02, State: Initialized, Role: FOLLOWER
I20260812 16:36:17.664372 10337 ts_tablet_manager.cc:1428] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:36:17.664938 10339 consensus_queue.cc:260] T 4b2e89d90762472f8c8244398adbbc35 P dcbf1ee01bc9412bba5a59422097fe02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f862f411754184980ef7f5c3566e97" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 36411 } } peers { permanent_uuid: "23f2fc19c97a446c981d92cf3ffe442e" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 39795 } } peers { permanent_uuid: "dcbf1ee01bc9412bba5a59422097fe02" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 37189 } }
I20260812 16:36:17.665256 10339 ts_tablet_manager.cc:1428] T 4b2e89d90762472f8c8244398adbbc35 P dcbf1ee01bc9412bba5a59422097fe02: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:36:17.665272 10012 heartbeater.cc:503] Master 127.19.238.126:43047 was elected leader, sending a full tablet report...
W20260812 16:36:17.695926 10163 tablet.cc:2369] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:36:17.720387 10348 raft_consensus.cc:482] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:36:17.720587 10348 raft_consensus.cc:504] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f862f411754184980ef7f5c3566e97" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 36411 } } peers { permanent_uuid: "23f2fc19c97a446c981d92cf3ffe442e" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 39795 } } peers { permanent_uuid: "dcbf1ee01bc9412bba5a59422097fe02" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 37189 } }
I20260812 16:36:17.721000 10348 leader_election.cc:288] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23f2fc19c97a446c981d92cf3ffe442e (127.19.238.66:39795), dcbf1ee01bc9412bba5a59422097fe02 (127.19.238.65:37189)
I20260812 16:36:17.725183 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.721331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:54729 (local address 127.19.238.66:39795)
0812 16:36:17.721400 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:17.721402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:17.721416 (+ 14us) server_negotiation.cc:407] Connection header received
0812 16:36:17.721452 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:17.721453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:17.721464 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:17.721553 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:17.721775 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.722538 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.724160 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.724769 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.724849 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.724858 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:17.725066 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 16:36:17.725108 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:36:17.725183 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.721535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:55977 (local address 127.19.238.65:37189)
0812 16:36:17.721623 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:17.721625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:17.721692 (+ 67us) server_negotiation.cc:407] Connection header received
0812 16:36:17.721761 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:17.721762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:17.721772 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:17.721835 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:17.721996 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.722692 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.724160 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.724769 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.724849 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.724858 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:17.725066 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 16:36:17.725108 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 16:36:17.725207 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.721245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:39795 (local address 127.19.238.67:54729)
0812 16:36:17.721313 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:17.721327 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:17.721382 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:17.721601 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:17.721603 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:17.721605 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:17.721690 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:17.721692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.722626 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.722627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:17.724041 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:17.724045 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.724872 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.724874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:17.724964 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.724967 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:36:17.725059 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 16:36:17.725125 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":26}
I20260812 16:36:17.725219 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.721393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:37189 (local address 127.19.238.67:55977)
0812 16:36:17.721614 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:17.721625 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:17.721689 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:17.721859 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:17.721860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:17.721861 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:17.721922 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:17.721923 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.722768 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.722768 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:17.724041 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:17.724045 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.724880 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.724881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:17.724963 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.724967 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:36:17.725039 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 16:36:17.725109 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":3,"thread_start_us":138,"threads_started":1}
I20260812 16:36:17.725641 10108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2e89d90762472f8c8244398adbbc35" candidate_uuid: "41f862f411754184980ef7f5c3566e97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23f2fc19c97a446c981d92cf3ffe442e" is_pre_election: true
I20260812 16:36:17.725835 10108 raft_consensus.cc:2454] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41f862f411754184980ef7f5c3566e97 in term 0.
I20260812 16:36:17.726090 9953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2e89d90762472f8c8244398adbbc35" candidate_uuid: "41f862f411754184980ef7f5c3566e97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcbf1ee01bc9412bba5a59422097fe02" is_pre_election: true
I20260812 16:36:17.726163 10187 leader_election.cc:302] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23f2fc19c97a446c981d92cf3ffe442e, 41f862f411754184980ef7f5c3566e97; no voters:
I20260812 16:36:17.726287 9953 raft_consensus.cc:2454] T 4b2e89d90762472f8c8244398adbbc35 P dcbf1ee01bc9412bba5a59422097fe02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41f862f411754184980ef7f5c3566e97 in term 0.
I20260812 16:36:17.726344 10348 raft_consensus.cc:2781] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:36:17.726401 10348 raft_consensus.cc:482] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:36:17.726431 10348 raft_consensus.cc:3037] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:17.727205 10348 raft_consensus.cc:504] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f862f411754184980ef7f5c3566e97" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 36411 } } peers { permanent_uuid: "23f2fc19c97a446c981d92cf3ffe442e" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 39795 } } peers { permanent_uuid: "dcbf1ee01bc9412bba5a59422097fe02" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 37189 } }
I20260812 16:36:17.727357 10348 leader_election.cc:288] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [CANDIDATE]: Term 1 election: Requested vote from peers 23f2fc19c97a446c981d92cf3ffe442e (127.19.238.66:39795), dcbf1ee01bc9412bba5a59422097fe02 (127.19.238.65:37189)
I20260812 16:36:17.727545 9953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2e89d90762472f8c8244398adbbc35" candidate_uuid: "41f862f411754184980ef7f5c3566e97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcbf1ee01bc9412bba5a59422097fe02"
I20260812 16:36:17.727546 10108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2e89d90762472f8c8244398adbbc35" candidate_uuid: "41f862f411754184980ef7f5c3566e97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23f2fc19c97a446c981d92cf3ffe442e"
I20260812 16:36:17.727638 10108 raft_consensus.cc:3037] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:17.727638 9953 raft_consensus.cc:3037] T 4b2e89d90762472f8c8244398adbbc35 P dcbf1ee01bc9412bba5a59422097fe02 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:17.728675 10108 raft_consensus.cc:2454] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41f862f411754184980ef7f5c3566e97 in term 1.
I20260812 16:36:17.728675 9953 raft_consensus.cc:2454] T 4b2e89d90762472f8c8244398adbbc35 P dcbf1ee01bc9412bba5a59422097fe02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41f862f411754184980ef7f5c3566e97 in term 1.
I20260812 16:36:17.728859 10187 leader_election.cc:302] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41f862f411754184980ef7f5c3566e97, dcbf1ee01bc9412bba5a59422097fe02; no voters:
I20260812 16:36:17.728983 10348 raft_consensus.cc:2781] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:17.729171 10348 raft_consensus.cc:686] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [term 1 LEADER]: Becoming Leader. State: Replica: 41f862f411754184980ef7f5c3566e97, State: Running, Role: LEADER
I20260812 16:36:17.729305 10348 consensus_queue.cc:237] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f862f411754184980ef7f5c3566e97" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 36411 } } peers { permanent_uuid: "23f2fc19c97a446c981d92cf3ffe442e" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 39795 } } peers { permanent_uuid: "dcbf1ee01bc9412bba5a59422097fe02" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 37189 } }
I20260812 16:36:17.730243 9789 catalog_manager.cc:5495] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41f862f411754184980ef7f5c3566e97 (127.19.238.67). New cstate: current_term: 1 leader_uuid: "41f862f411754184980ef7f5c3566e97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f862f411754184980ef7f5c3566e97" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 36411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23f2fc19c97a446c981d92cf3ffe442e" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 39795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcbf1ee01bc9412bba5a59422097fe02" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 37189 } health_report { overall_health: UNKNOWN } } }
I20260812 16:36:17.749410 10333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.746966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49102 (local address 127.19.238.67:36411)
0812 16:36:17.747069 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:17.747071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:17.747079 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:36:17.747120 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:17.747120 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:17.747138 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:17.747203 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:36:17.747386 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.747942 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.748708 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.748887 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.748937 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.749078 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:36:17.749179 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:36:17.749240 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:17.749314 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 16:36:17.749331 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 16:36:17.751562 10108 raft_consensus.cc:1261] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e [term 1 FOLLOWER]: Refusing update from remote peer 41f862f411754184980ef7f5c3566e97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:36:17.751576 9953 raft_consensus.cc:1261] T 4b2e89d90762472f8c8244398adbbc35 P dcbf1ee01bc9412bba5a59422097fe02 [term 1 FOLLOWER]: Refusing update from remote peer 41f862f411754184980ef7f5c3566e97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:36:17.751998 10348 consensus_queue.cc:1035] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23f2fc19c97a446c981d92cf3ffe442e" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 39795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:36:17.752173 10351 consensus_queue.cc:1035] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcbf1ee01bc9412bba5a59422097fe02" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 37189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:36:17.755496 10357 mvcc.cc:205] Tried to move back new op lower bound from 7317719358466965504 to 7317719358380048384. Current Snapshot: MvccSnapshot[applied={T|T < 7317719358466965504}]
I20260812 16:36:17.757179 9789 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:36:17.757292 9789 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:36:17.758111 9789 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 893e04f9038344ee8c2b5b5aa2ceb716: Sending DeleteTablet for 3 replicas of tablet 4b2e89d90762472f8c8244398adbbc35
I20260812 16:36:17.758535 10084 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b2e89d90762472f8c8244398adbbc35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:36:17 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:42634
I20260812 16:36:17.758530 9932 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b2e89d90762472f8c8244398adbbc35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:36:17 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:33056
I20260812 16:36:17.758683 10241 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b2e89d90762472f8c8244398adbbc35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:36:17 UTC) from {username='kudu', principal='kudu/127.19.238.126@KRBTEST.COM'} at 127.0.0.1:49092
I20260812 16:36:17.758792 10367 tablet_replica.cc:330] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e: stopping tablet replica
I20260812 16:36:17.758894 10367 raft_consensus.cc:2229] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:36:17.759001 10367 raft_consensus.cc:2258] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:36:17.759114 10368 tablet_replica.cc:330] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97: stopping tablet replica
I20260812 16:36:17.759209 10368 raft_consensus.cc:2229] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:36:17.759290 10368 raft_consensus.cc:2258] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:36:17.759284 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 9862
I20260812 16:36:17.761926 10367 ts_tablet_manager.cc:1905] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:36:17.762111 10368 ts_tablet_manager.cc:1905] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:36:17.763926 10367 ts_tablet_manager.cc:1918] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:36:17.763965 10368 ts_tablet_manager.cc:1918] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:36:17.764019 10367 log.cc:1196] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/4b2e89d90762472f8c8244398adbbc35
I20260812 16:36:17.764035 10368 log.cc:1196] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/4b2e89d90762472f8c8244398adbbc35
I20260812 16:36:17.764349 10368 ts_tablet_manager.cc:1939] T 4b2e89d90762472f8c8244398adbbc35 P 41f862f411754184980ef7f5c3566e97: Deleting consensus metadata
I20260812 16:36:17.764362 10367 ts_tablet_manager.cc:1939] T 4b2e89d90762472f8c8244398adbbc35 P 23f2fc19c97a446c981d92cf3ffe442e: Deleting consensus metadata
I20260812 16:36:17.764760 9768 catalog_manager.cc:4841] TS 41f862f411754184980ef7f5c3566e97 (127.19.238.67:36411): tablet 4b2e89d90762472f8c8244398adbbc35 (table test-table [id=65bb2065acdb484b9b7ef72d5cfcbe3c]) successfully deleted
I20260812 16:36:17.764822 9769 catalog_manager.cc:4841] TS 23f2fc19c97a446c981d92cf3ffe442e (127.19.238.66:39795): tablet 4b2e89d90762472f8c8244398adbbc35 (table test-table [id=65bb2065acdb484b9b7ef72d5cfcbe3c]) successfully deleted
W20260812 16:36:17.769354 9769 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.238.65:37189 (error 108)
I20260812 16:36:17.769591 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 10017
I20260812 16:36:17.777521 10332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.777356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:37189 (local address 127.0.0.1:33072)
0812 16:36:17.777432 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:17.777481 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:37189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 16:36:17.777808 9769 catalog_manager.cc:4584] TS dcbf1ee01bc9412bba5a59422097fe02 (127.19.238.65:37189): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4b2e89d90762472f8c8244398adbbc35: Network error: Client connection negotiation failed: client connection to 127.19.238.65:37189: connect: Connection refused (error 111)
I20260812 16:36:17.782536 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 10168
I20260812 16:36:17.790249 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 9750
2026-08-12T16:36:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2087 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T16:36:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:17Z Disabled control of system clock
I20260812 16:36:17.822839 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:32823
--webserver_interface=127.19.238.126
--webserver_port=0
--builtin_ntp_servers=127.19.238.84:36127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:32823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:36:17.910303 10373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:17.910552 10373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:17.910576 10373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:17.911684 10373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:17.911721 10373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:17.911734 10373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:17.911746 10373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:17.911758 10373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:17.913139 10373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:36127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:32823
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:32823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:17.913350 10373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:17.913606 10373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:17.918612 10379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:17.918752 10382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:17.918785 10380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:17.919312 10373 server_base.cc:1034] running on GCE node
I20260812 16:36:17.919555 10373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:17.919862 10373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:17.921056 10373 hybrid_clock.cc:648] HybridClock initialized: now 1786552577921029 us; error 48 us; skew 500 ppm
I20260812 16:36:17.922358 10373 webserver.cc:460] Webserver started at http://127.19.238.126:41807/ using document root <none> and password file <none>
I20260812 16:36:17.922595 10373 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:17.922637 10373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:17.922734 10373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:17.923789 10373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "d63dc2d5f65b462d9e727f052c4743fa"
format_stamp: "Formatted at 2026-08-12 16:36:17 on dist-test-slave-mvvj"
server_key: "3fdfa6dd9102d22c6188d4c89ecc6638"
server_key_iv: "0ff0345c4edd5ed0794d187bd77c5091"
server_key_version: "encryptionkey@0"
I20260812 16:36:17.924175 10373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "d63dc2d5f65b462d9e727f052c4743fa"
format_stamp: "Formatted at 2026-08-12 16:36:17 on dist-test-slave-mvvj"
server_key: "3fdfa6dd9102d22c6188d4c89ecc6638"
server_key_iv: "0ff0345c4edd5ed0794d187bd77c5091"
server_key_version: "encryptionkey@0"
I20260812 16:36:17.925460 10373 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:17.926198 10388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:17.926381 10373 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:17.926473 10373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "d63dc2d5f65b462d9e727f052c4743fa"
format_stamp: "Formatted at 2026-08-12 16:36:17 on dist-test-slave-mvvj"
server_key: "3fdfa6dd9102d22c6188d4c89ecc6638"
server_key_iv: "0ff0345c4edd5ed0794d187bd77c5091"
server_key_version: "encryptionkey@0"
I20260812 16:36:17.926558 10373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:17.946331 10373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:17.946730 10373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:17.946906 10373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:17.951072 10373 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:32823
I20260812 16:36:17.951112 10448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:32823 every 8 connection(s)
I20260812 16:36:17.951489 10373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:17.952540 10449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:17.954896 10449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa: Bootstrap starting.
I20260812 16:36:17.955529 10449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:17.955852 10449 log.cc:824] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:17.956561 10449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa: No bootstrap required, opened a new log
I20260812 16:36:17.958328 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 10373
I20260812 16:36:17.958448 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:17.958279 10449 raft_consensus.cc:348] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63dc2d5f65b462d9e727f052c4743fa" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 32823 } }
I20260812 16:36:17.958482 10449 raft_consensus.cc:374] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:17.958519 10449 raft_consensus.cc:729] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63dc2d5f65b462d9e727f052c4743fa, State: Initialized, Role: FOLLOWER
I20260812 16:36:17.958720 20409 external_mini_cluster.cc:1428] Setting key 15f58cf7bb28f8064ba2fee2b4e64c12
I20260812 16:36:17.958701 10449 consensus_queue.cc:260] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63dc2d5f65b462d9e727f052c4743fa" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 32823 } }
I20260812 16:36:17.958786 10449 raft_consensus.cc:388] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:17.958819 10449 raft_consensus.cc:482] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:17.958874 10449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:17.959599 10449 raft_consensus.cc:504] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63dc2d5f65b462d9e727f052c4743fa" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 32823 } }
I20260812 16:36:17.959689 10449 leader_election.cc:302] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d63dc2d5f65b462d9e727f052c4743fa; no voters:
I20260812 16:36:17.959951 10449 leader_election.cc:288] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:17.959975 10454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:17.960119 10454 raft_consensus.cc:686] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa [term 1 LEADER]: Becoming Leader. State: Replica: d63dc2d5f65b462d9e727f052c4743fa, State: Running, Role: LEADER
I20260812 16:36:17.960234 10449 sys_catalog.cc:564] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:17.960206 10454 consensus_queue.cc:237] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63dc2d5f65b462d9e727f052c4743fa" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 32823 } }
I20260812 16:36:17.960587 10456 sys_catalog.cc:455] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d63dc2d5f65b462d9e727f052c4743fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63dc2d5f65b462d9e727f052c4743fa" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 32823 } } }
I20260812 16:36:17.960605 10457 sys_catalog.cc:455] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader d63dc2d5f65b462d9e727f052c4743fa. Latest consensus state: current_term: 1 leader_uuid: "d63dc2d5f65b462d9e727f052c4743fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63dc2d5f65b462d9e727f052c4743fa" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 32823 } } }
I20260812 16:36:17.960676 10456 sys_catalog.cc:458] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:17.960707 10457 sys_catalog.cc:458] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:17.960953 10459 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:17.961495 10459 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:17.962862 10459 catalog_manager.cc:1347] Generated new cluster ID: fb319bf1a85b4d459c2980c1e2db4de4
I20260812 16:36:17.962924 10459 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:17.966010 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:17.959256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37824 (local address 127.19.238.126:32823)
0812 16:36:17.959502 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:17.959516 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:17.959541 (+ 25us) server_negotiation.cc:407] Connection header received
0812 16:36:17.959638 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:17.959643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:17.959751 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:17.959926 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:17.960299 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.962257 (+ 1958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.963521 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:17.964126 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:17.964241 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:17.965253 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:17.965257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:17.965258 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:17.965489 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:17.965535 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:17.965561 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:17.965635 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:17.965707 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 16:36:17.965792 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":110,"threads_started":1}
I20260812 16:36:17.972497 10459 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:17.973201 10459 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:17.982733 10459 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa: Generated new TSK 0
I20260812 16:36:17.982954 10459 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:17.984967 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.65:0
--local_ip_for_outbound_sockets=127.19.238.65
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:32823
--builtin_ntp_servers=127.19.238.84:36127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:36:18.079728 10475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:18.080054 10475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:18.080081 10475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:18.081461 10475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:18.081506 10475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:18.081535 10475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.65
I20260812 16:36:18.083048 10475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:36127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.238.65
--webserver_port=0
--tserver_master_addrs=127.19.238.126:32823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.65
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:18.083330 10475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:18.083685 10475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:18.089316 10480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:18.089442 10481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:18.089567 10483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:18.089694 10475 server_base.cc:1034] running on GCE node
I20260812 16:36:18.089963 10475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:18.090234 10475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:18.091414 10475 hybrid_clock.cc:648] HybridClock initialized: now 1786552578091381 us; error 45 us; skew 500 ppm
I20260812 16:36:18.092871 10475 webserver.cc:460] Webserver started at http://127.19.238.65:35303/ using document root <none> and password file <none>
I20260812 16:36:18.093143 10475 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:18.093218 10475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:18.093384 10475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:18.094563 10475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-0/data/instance:
uuid: "6e5848bdb7c84a8394090972ea79fed9"
format_stamp: "Formatted at 2026-08-12 16:36:18 on dist-test-slave-mvvj"
server_key: "01d801632474473540dbb7b25de9ad36"
server_key_iv: "25fd8aa1347979dc4cf789338d37fdf5"
server_key_version: "encryptionkey@0"
I20260812 16:36:18.094955 10475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance:
uuid: "6e5848bdb7c84a8394090972ea79fed9"
format_stamp: "Formatted at 2026-08-12 16:36:18 on dist-test-slave-mvvj"
server_key: "01d801632474473540dbb7b25de9ad36"
server_key_iv: "25fd8aa1347979dc4cf789338d37fdf5"
server_key_version: "encryptionkey@0"
I20260812 16:36:18.096467 10475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 16:36:18.097360 10490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:18.098874 10475 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 16:36:18.098960 10475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-0/wal
uuid: "6e5848bdb7c84a8394090972ea79fed9"
format_stamp: "Formatted at 2026-08-12 16:36:18 on dist-test-slave-mvvj"
server_key: "01d801632474473540dbb7b25de9ad36"
server_key_iv: "25fd8aa1347979dc4cf789338d37fdf5"
server_key_version: "encryptionkey@0"
I20260812 16:36:18.099020 10475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:18.120806 10475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:18.121135 10475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:18.121245 10475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:18.121519 10475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:36:18.121840 10475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:36:18.121881 10475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:18.121932 10475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:36:18.121964 10475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:18.128558 10475 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.65:43427
I20260812 16:36:18.128624 10629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.65:43427 every 8 connection(s)
I20260812 16:36:18.128984 10475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-0/data/info.pb
I20260812 16:36:18.131671 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 10475
I20260812 16:36:18.131803 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-0/wal/instance
I20260812 16:36:18.132040 20409 external_mini_cluster.cc:1428] Setting key 2bf22b490e5e6d1f6af19d9877c3871c
I20260812 16:36:18.133198 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.66:0
--local_ip_for_outbound_sockets=127.19.238.66
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:32823
--builtin_ntp_servers=127.19.238.84:36127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 16:36:18.135861 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:18.129810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:34969 (local address 127.19.238.126:32823)
0812 16:36:18.129904 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:18.129906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:18.130117 (+ 211us) server_negotiation.cc:407] Connection header received
0812 16:36:18.130799 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:18.130801 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:18.130817 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:18.130904 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:18.131663 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.132461 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.134022 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.134217 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.134278 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:18.135367 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:18.135369 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:18.135369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:18.135386 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:18.135465 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:18.135471 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:18.135541 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:18.135677 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 16:36:18.135717 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 16:36:18.136085 10632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:18.129771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:32823 (local address 127.19.238.65:34969)
0812 16:36:18.129988 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:18.130023 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:18.130680 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:18.131002 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:18.131005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:18.131014 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:18.131558 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:18.131564 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.132582 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.132583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:18.133893 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:18.133897 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.134313 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.134314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:18.134478 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:18.135208 (+ 730us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:36:18.135208 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:18.135239 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:36:18.135265 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:18.135570 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:18.135577 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:18.135580 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:36:18.135656 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 16:36:18.135749 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
I20260812 16:36:18.136690 10630 heartbeater.cc:347] Connected to a master server at 127.19.238.126:32823
I20260812 16:36:18.136799 10630 heartbeater.cc:464] Registering TS with master...
I20260812 16:36:18.137001 10630 heartbeater.cc:511] Master 127.19.238.126:32823 requested a full tablet report, sending...
I20260812 16:36:18.137557 10406 ts_manager.cc:194] Registered new tserver with Master: 6e5848bdb7c84a8394090972ea79fed9 (127.19.238.65:43427)
I20260812 16:36:18.138269 10406 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.238.65:34969
W20260812 16:36:18.217902 10633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:18.218129 10633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:18.218152 10633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:18.219380 10633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:18.219431 10633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:18.219457 10633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.66
I20260812 16:36:18.220991 10633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:36127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.238.66
--webserver_port=0
--tserver_master_addrs=127.19.238.126:32823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.66
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:18.221246 10633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:18.221573 10633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:18.226861 10639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:18.226861 10642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:18.226902 10640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:18.227057 10633 server_base.cc:1034] running on GCE node
I20260812 16:36:18.227280 10633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:18.227488 10633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:18.228636 10633 hybrid_clock.cc:648] HybridClock initialized: now 1786552578228615 us; error 36 us; skew 500 ppm
I20260812 16:36:18.230075 10633 webserver.cc:460] Webserver started at http://127.19.238.66:42741/ using document root <none> and password file <none>
I20260812 16:36:18.230355 10633 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:18.230405 10633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:18.230567 10633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:18.231734 10633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-1/data/instance:
uuid: "8757ea26513a41cdbba0cec7eff60ced"
format_stamp: "Formatted at 2026-08-12 16:36:18 on dist-test-slave-mvvj"
server_key: "43914749d191306842fe02887d651c05"
server_key_iv: "75e59f00dd7837255c43122f0679a440"
server_key_version: "encryptionkey@0"
I20260812 16:36:18.232168 10633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance:
uuid: "8757ea26513a41cdbba0cec7eff60ced"
format_stamp: "Formatted at 2026-08-12 16:36:18 on dist-test-slave-mvvj"
server_key: "43914749d191306842fe02887d651c05"
server_key_iv: "75e59f00dd7837255c43122f0679a440"
server_key_version: "encryptionkey@0"
I20260812 16:36:18.233453 10633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:18.234148 10650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:18.234303 10633 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:36:18.234418 10633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-1/wal
uuid: "8757ea26513a41cdbba0cec7eff60ced"
format_stamp: "Formatted at 2026-08-12 16:36:18 on dist-test-slave-mvvj"
server_key: "43914749d191306842fe02887d651c05"
server_key_iv: "75e59f00dd7837255c43122f0679a440"
server_key_version: "encryptionkey@0"
I20260812 16:36:18.234499 10633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:18.247249 10633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:18.247605 10633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:18.247764 10633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:18.248106 10633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:36:18.248533 10633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:36:18.248581 10633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:18.248624 10633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:36:18.248651 10633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:18.255976 10633 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.66:41965
I20260812 16:36:18.256026 10782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.66:41965 every 8 connection(s)
I20260812 16:36:18.256484 10633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-1/data/info.pb
I20260812 16:36:18.259898 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 10633
I20260812 16:36:18.260030 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-1/wal/instance
I20260812 16:36:18.260267 20409 external_mini_cluster.cc:1428] Setting key 69bb6d63fbbb1a4268d428a2574f362f
I20260812 16:36:18.261431 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.238.67:0
--local_ip_for_outbound_sockets=127.19.238.67
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:32823
--builtin_ntp_servers=127.19.238.84:36127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 16:36:18.263813 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:18.257926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.66:51337 (local address 127.19.238.126:32823)
0812 16:36:18.258014 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:18.258017 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:18.258298 (+ 281us) server_negotiation.cc:407] Connection header received
0812 16:36:18.258824 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:18.258826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:18.258839 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:18.258887 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:18.259725 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.260536 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.262153 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.262345 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.262412 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:18.263466 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:18.263467 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:18.263468 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:18.263472 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:18.263488 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:18.263492 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:18.263550 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:18.263670 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 16:36:18.263714 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 16:36:18.264127 10786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:18.257936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:32823 (local address 127.19.238.66:51337)
0812 16:36:18.258173 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:18.258210 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:18.258704 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:18.258982 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:18.258986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:18.259004 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:18.259594 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:18.259602 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.260690 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.260691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:18.262064 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:18.262068 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.262441 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.262442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:18.262620 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:18.263244 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:36:18.263245 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:18.263293 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:36:18.263333 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:18.263569 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:18.263576 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:18.263579 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:36:18.263655 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 16:36:18.263862 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":134,"threads_started":1}
I20260812 16:36:18.265853 10783 heartbeater.cc:347] Connected to a master server at 127.19.238.126:32823
I20260812 16:36:18.265972 10783 heartbeater.cc:464] Registering TS with master...
I20260812 16:36:18.266188 10783 heartbeater.cc:511] Master 127.19.238.126:32823 requested a full tablet report, sending...
I20260812 16:36:18.266676 10406 ts_manager.cc:194] Registered new tserver with Master: 8757ea26513a41cdbba0cec7eff60ced (127.19.238.66:41965)
I20260812 16:36:18.267233 10406 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.238.66:51337
W20260812 16:36:18.357667 10787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:18.357915 10787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:18.357937 10787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:18.359100 10787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:18.359144 10787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:18.359169 10787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.238.67
I20260812 16:36:18.360692 10787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:36127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.238.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.238.67
--webserver_port=0
--tserver_master_addrs=127.19.238.126:32823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.238.67
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:18.360921 10787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:18.361218 10787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:18.366425 10795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:18.366438 10793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:18.366469 10792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:18.366848 10787 server_base.cc:1034] running on GCE node
I20260812 16:36:18.367089 10787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:18.367352 10787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:18.368537 10787 hybrid_clock.cc:648] HybridClock initialized: now 1786552578368516 us; error 31 us; skew 500 ppm
I20260812 16:36:18.369987 10787 webserver.cc:460] Webserver started at http://127.19.238.67:40455/ using document root <none> and password file <none>
I20260812 16:36:18.370261 10787 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:18.370338 10787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:18.370556 10787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:18.371747 10787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-2/data/instance:
uuid: "588c11fd81bf448f9a31c562b9e3702e"
format_stamp: "Formatted at 2026-08-12 16:36:18 on dist-test-slave-mvvj"
server_key: "9a396a7aa8011d4fa23ab8aaa527de10"
server_key_iv: "ef18d34f4007cf1e7f37bf4130eb5f6a"
server_key_version: "encryptionkey@0"
I20260812 16:36:18.372242 10787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance:
uuid: "588c11fd81bf448f9a31c562b9e3702e"
format_stamp: "Formatted at 2026-08-12 16:36:18 on dist-test-slave-mvvj"
server_key: "9a396a7aa8011d4fa23ab8aaa527de10"
server_key_iv: "ef18d34f4007cf1e7f37bf4130eb5f6a"
server_key_version: "encryptionkey@0"
I20260812 16:36:18.373682 10787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:36:18.374469 10803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:18.374673 10787 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:18.374792 10787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-2/wal
uuid: "588c11fd81bf448f9a31c562b9e3702e"
format_stamp: "Formatted at 2026-08-12 16:36:18 on dist-test-slave-mvvj"
server_key: "9a396a7aa8011d4fa23ab8aaa527de10"
server_key_iv: "ef18d34f4007cf1e7f37bf4130eb5f6a"
server_key_version: "encryptionkey@0"
I20260812 16:36:18.374874 10787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:18.385113 10787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:18.385437 10787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:18.385605 10787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:18.385854 10787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:36:18.386204 10787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:36:18.386264 10787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:18.386312 10787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:36:18.386339 10787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:18.392721 10787 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.67:45297
I20260812 16:36:18.392758 10934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.67:45297 every 8 connection(s)
I20260812 16:36:18.393129 10787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-2/data/info.pb
I20260812 16:36:18.398587 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:18.393951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.67:39153 (local address 127.19.238.126:32823)
0812 16:36:18.394009 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:18.394012 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:18.394278 (+ 266us) server_negotiation.cc:407] Connection header received
0812 16:36:18.394773 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:18.394775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:18.394786 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:18.394847 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:18.395513 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.396136 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.397148 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.397345 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.397446 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:18.398280 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:18.398281 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:18.398282 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:18.398287 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:18.398321 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:18.398326 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:18.398380 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:18.398493 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 16:36:18.398519 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:36:18.398753 10937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:18.393957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:32823 (local address 127.19.238.67:39153)
0812 16:36:18.394153 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:18.394188 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:18.394679 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:18.394920 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:18.394924 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:18.394932 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:18.395434 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:18.395438 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.396224 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.396225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:18.397067 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:18.397069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.397419 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.397420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:18.397548 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:18.398130 (+ 582us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:36:18.398130 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:18.398160 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:36:18.398183 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:18.398394 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:18.398398 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:18.398401 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:36:18.398456 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 16:36:18.398553 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
I20260812 16:36:18.399225 10935 heartbeater.cc:347] Connected to a master server at 127.19.238.126:32823
I20260812 16:36:18.399317 10935 heartbeater.cc:464] Registering TS with master...
I20260812 16:36:18.399456 10935 heartbeater.cc:511] Master 127.19.238.126:32823 requested a full tablet report, sending...
I20260812 16:36:18.399895 10406 ts_manager.cc:194] Registered new tserver with Master: 588c11fd81bf448f9a31c562b9e3702e (127.19.238.67:45297)
I20260812 16:36:18.400238 10406 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.238.67:39153
I20260812 16:36:18.400543 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 10787
I20260812 16:36:18.400653 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-2/wal/instance
I20260812 16:36:18.400882 20409 external_mini_cluster.cc:1428] Setting key b0134050822b3765881092808f0df43a
I20260812 16:36:18.401860 20409 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:36:18.406279 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:18.403302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37840 (local address 127.19.238.126:32823)
0812 16:36:18.403415 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:18.403417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:18.403472 (+ 55us) server_negotiation.cc:407] Connection header received
0812 16:36:18.403549 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:18.403550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:18.403564 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:18.403633 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:18.403803 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.404321 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.405076 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.405263 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.405346 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:18.406003 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:18.406004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:18.406005 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:18.406009 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:18.406022 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:18.406026 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:18.406109 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:18.406172 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 16:36:18.406194 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 16:36:18.407794 10406 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:36:18.409365 10406 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:36:18.416347 10947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:18.411921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35728 (local address 127.19.238.66:41965)
0812 16:36:18.412352 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:18.412356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:18.412365 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:36:18.412402 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:18.412404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:18.412418 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:18.412647 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:18.412707 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.413886 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.415137 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.415960 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.416035 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:18.416129 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:18.416243 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 16:36:18.416273 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":108,"threads_started":1}
I20260812 16:36:18.416352 10944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:18.411757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:41965 (local address 127.0.0.1:35728)
0812 16:36:18.412130 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:18.412140 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:18.412200 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:18.412515 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:18.412516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:18.412518 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:18.412566 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:18.412568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.413983 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.413984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:18.415030 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:18.415032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.416038 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.416039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:18.416122 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:18.416137 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 16:36:18.416216 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 16:36:18.416270 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":132,"threads_started":1}
I20260812 16:36:18.417014 10948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:18.412030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:45297 (local address 127.0.0.1:53650)
0812 16:36:18.412283 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:18.412292 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:18.412406 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:18.412728 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:18.412729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:18.412731 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:18.412780 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:18.412782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.413651 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.413652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:18.414995 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:18.414999 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.416754 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.416755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:18.416830 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:18.416835 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:36:18.416885 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 16:36:18.416929 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":67,"threads_started":1}
I20260812 16:36:18.417114 10946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:18.411867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49430 (local address 127.19.238.65:43427)
0812 16:36:18.412264 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:18.412270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:18.412281 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:36:18.412337 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:18.412340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:18.412357 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:18.412647 (+ 290us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:18.412709 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.413893 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.415933 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.416708 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.416811 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:18.416903 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:18.416983 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 16:36:18.417022 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":76,"threads_started":1}
I20260812 16:36:18.417121 10945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:18.411723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:43427 (local address 127.0.0.1:49430)
0812 16:36:18.411988 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:18.412034 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:18.412106 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:18.412479 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:18.412482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:18.412485 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:18.412566 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:18.412568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.415121 (+ 2553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.415122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:18.415781 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:18.415782 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.416833 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.416834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:18.416905 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:18.416909 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:36:18.416972 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 16:36:18.417016 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":8,"thread_start_us":121,"threads_started":1}
I20260812 16:36:18.417325 10949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:18.412140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53650 (local address 127.19.238.67:45297)
0812 16:36:18.412566 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:18.412570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:18.412586 (+ 16us) server_negotiation.cc:407] Connection header received
0812 16:36:18.412619 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:18.412620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:18.412632 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:18.412719 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:18.412837 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.413550 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.415143 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.415949 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.416036 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:18.416129 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:18.417188 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0812 16:36:18.417233 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":82,"threads_started":1}
I20260812 16:36:18.417222 10704 tablet_service.cc:1467] Processing CreateTablet for tablet 7b8859d22d764e3eaa800d0cb4e5177e (DEFAULT_TABLE table=test-table [id=f9383cd49d7545b281079c08983d66cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:36:18.417694 10704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b8859d22d764e3eaa800d0cb4e5177e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:18.417716 10546 tablet_service.cc:1467] Processing CreateTablet for tablet 7b8859d22d764e3eaa800d0cb4e5177e (DEFAULT_TABLE table=test-table [id=f9383cd49d7545b281079c08983d66cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:36:18.418066 10546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b8859d22d764e3eaa800d0cb4e5177e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:18.420383 10859 tablet_service.cc:1467] Processing CreateTablet for tablet 7b8859d22d764e3eaa800d0cb4e5177e (DEFAULT_TABLE table=test-table [id=f9383cd49d7545b281079c08983d66cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:36:18.420493 10950 tablet_bootstrap.cc:492] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced: Bootstrap starting.
I20260812 16:36:18.420738 10859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b8859d22d764e3eaa800d0cb4e5177e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:18.421111 10951 tablet_bootstrap.cc:492] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9: Bootstrap starting.
I20260812 16:36:18.421211 10950 tablet_bootstrap.cc:654] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:18.421446 10950 log.cc:824] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:18.421751 10951 tablet_bootstrap.cc:654] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:18.422048 10951 log.cc:824] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:18.422391 10950 tablet_bootstrap.cc:492] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced: No bootstrap required, opened a new log
I20260812 16:36:18.422472 10950 ts_tablet_manager.cc:1397] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:36:18.422865 10951 tablet_bootstrap.cc:492] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9: No bootstrap required, opened a new log
I20260812 16:36:18.422945 10951 ts_tablet_manager.cc:1397] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:36:18.424352 10954 tablet_bootstrap.cc:492] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e: Bootstrap starting.
I20260812 16:36:18.424803 10950 raft_consensus.cc:348] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5848bdb7c84a8394090972ea79fed9" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43427 } } peers { permanent_uuid: "8757ea26513a41cdbba0cec7eff60ced" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41965 } } peers { permanent_uuid: "588c11fd81bf448f9a31c562b9e3702e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45297 } }
I20260812 16:36:18.425031 10950 raft_consensus.cc:374] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:18.425086 10950 raft_consensus.cc:729] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8757ea26513a41cdbba0cec7eff60ced, State: Initialized, Role: FOLLOWER
I20260812 16:36:18.425195 10954 tablet_bootstrap.cc:654] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:18.425225 10950 consensus_queue.cc:260] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5848bdb7c84a8394090972ea79fed9" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43427 } } peers { permanent_uuid: "8757ea26513a41cdbba0cec7eff60ced" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41965 } } peers { permanent_uuid: "588c11fd81bf448f9a31c562b9e3702e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45297 } }
I20260812 16:36:18.425522 10950 ts_tablet_manager.cc:1428] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:36:18.425562 10783 heartbeater.cc:503] Master 127.19.238.126:32823 was elected leader, sending a full tablet report...
I20260812 16:36:18.425581 10951 raft_consensus.cc:348] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5848bdb7c84a8394090972ea79fed9" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43427 } } peers { permanent_uuid: "8757ea26513a41cdbba0cec7eff60ced" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41965 } } peers { permanent_uuid: "588c11fd81bf448f9a31c562b9e3702e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45297 } }
I20260812 16:36:18.425757 10951 raft_consensus.cc:374] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:18.425803 10951 raft_consensus.cc:729] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e5848bdb7c84a8394090972ea79fed9, State: Initialized, Role: FOLLOWER
I20260812 16:36:18.425518 10954 log.cc:824] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:18.425942 10951 consensus_queue.cc:260] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5848bdb7c84a8394090972ea79fed9" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43427 } } peers { permanent_uuid: "8757ea26513a41cdbba0cec7eff60ced" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41965 } } peers { permanent_uuid: "588c11fd81bf448f9a31c562b9e3702e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45297 } }
I20260812 16:36:18.426198 10951 ts_tablet_manager.cc:1428] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260812 16:36:18.426260 10630 heartbeater.cc:503] Master 127.19.238.126:32823 was elected leader, sending a full tablet report...
I20260812 16:36:18.426800 10954 tablet_bootstrap.cc:492] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e: No bootstrap required, opened a new log
I20260812 16:36:18.426882 10954 ts_tablet_manager.cc:1397] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:36:18.428901 10954 raft_consensus.cc:348] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5848bdb7c84a8394090972ea79fed9" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43427 } } peers { permanent_uuid: "8757ea26513a41cdbba0cec7eff60ced" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41965 } } peers { permanent_uuid: "588c11fd81bf448f9a31c562b9e3702e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45297 } }
I20260812 16:36:18.429052 10954 raft_consensus.cc:374] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:18.429075 10954 raft_consensus.cc:729] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 588c11fd81bf448f9a31c562b9e3702e, State: Initialized, Role: FOLLOWER
I20260812 16:36:18.429193 10954 consensus_queue.cc:260] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5848bdb7c84a8394090972ea79fed9" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43427 } } peers { permanent_uuid: "8757ea26513a41cdbba0cec7eff60ced" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41965 } } peers { permanent_uuid: "588c11fd81bf448f9a31c562b9e3702e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45297 } }
I20260812 16:36:18.429462 10954 ts_tablet_manager.cc:1428] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:36:18.429577 10935 heartbeater.cc:503] Master 127.19.238.126:32823 was elected leader, sending a full tablet report...
I20260812 16:36:18.473248 10957 raft_consensus.cc:482] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:36:18.473415 10957 raft_consensus.cc:504] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5848bdb7c84a8394090972ea79fed9" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43427 } } peers { permanent_uuid: "8757ea26513a41cdbba0cec7eff60ced" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41965 } } peers { permanent_uuid: "588c11fd81bf448f9a31c562b9e3702e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45297 } }
I20260812 16:36:18.473838 10957 leader_election.cc:288] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8757ea26513a41cdbba0cec7eff60ced (127.19.238.66:41965), 588c11fd81bf448f9a31c562b9e3702e (127.19.238.67:45297)
I20260812 16:36:18.476890 10632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:18.474033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.66:41965 (local address 127.19.238.65:41793)
0812 16:36:18.474095 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:18.474108 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:18.474172 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:18.474381 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:18.474381 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:18.474383 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:18.474479 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:18.474481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.475240 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.475242 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:18.476109 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:18.476112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.476663 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.476664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:18.476739 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:18.476743 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 16:36:18.476797 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 16:36:18.476836 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 16:36:18.476959 10947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:18.474186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:41793 (local address 127.19.238.66:41965)
0812 16:36:18.474257 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:18.474259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:18.474266 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:36:18.474294 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:18.474294 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:18.474301 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:18.474361 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:18.474563 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.475161 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.476193 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.476584 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.476641 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:18.476648 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:18.476811 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 16:36:18.476854 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 16:36:18.477056 10959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:18.474033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.67:45297 (local address 127.19.238.65:42589)
0812 16:36:18.474306 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:18.474315 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:18.474395 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:18.474614 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:18.474615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:18.474617 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:36:18.474683 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:18.474685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.475384 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.475385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:18.476211 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:18.476213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.476763 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.476764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:18.476866 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:18.476869 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 16:36:18.476932 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 16:36:18.476965 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":94,"thread_start_us":102,"threads_started":1}
I20260812 16:36:18.477088 10949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:18.474142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.238.65:42589 (local address 127.19.238.67:45297)
0812 16:36:18.474198 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:18.474201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:18.474383 (+ 182us) server_negotiation.cc:407] Connection header received
0812 16:36:18.474500 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:18.474502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:18.474520 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:18.474572 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:36:18.474767 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.475299 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.476299 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.476686 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.476754 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:18.476762 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:18.476948 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 16:36:18.476997 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 16:36:18.477262 10732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b8859d22d764e3eaa800d0cb4e5177e" candidate_uuid: "6e5848bdb7c84a8394090972ea79fed9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8757ea26513a41cdbba0cec7eff60ced" is_pre_election: true
I20260812 16:36:18.477408 10732 raft_consensus.cc:2454] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e5848bdb7c84a8394090972ea79fed9 in term 0.
I20260812 16:36:18.477391 10883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b8859d22d764e3eaa800d0cb4e5177e" candidate_uuid: "6e5848bdb7c84a8394090972ea79fed9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "588c11fd81bf448f9a31c562b9e3702e" is_pre_election: true
I20260812 16:36:18.477545 10883 raft_consensus.cc:2454] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e5848bdb7c84a8394090972ea79fed9 in term 0.
I20260812 16:36:18.477638 10494 leader_election.cc:302] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e5848bdb7c84a8394090972ea79fed9, 8757ea26513a41cdbba0cec7eff60ced; no voters:
I20260812 16:36:18.477773 10957 raft_consensus.cc:2781] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:36:18.477849 10957 raft_consensus.cc:482] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:36:18.477885 10957 raft_consensus.cc:3037] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:18.478639 10957 raft_consensus.cc:504] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5848bdb7c84a8394090972ea79fed9" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43427 } } peers { permanent_uuid: "8757ea26513a41cdbba0cec7eff60ced" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41965 } } peers { permanent_uuid: "588c11fd81bf448f9a31c562b9e3702e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45297 } }
I20260812 16:36:18.478791 10957 leader_election.cc:288] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 [CANDIDATE]: Term 1 election: Requested vote from peers 8757ea26513a41cdbba0cec7eff60ced (127.19.238.66:41965), 588c11fd81bf448f9a31c562b9e3702e (127.19.238.67:45297)
I20260812 16:36:18.478972 10732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b8859d22d764e3eaa800d0cb4e5177e" candidate_uuid: "6e5848bdb7c84a8394090972ea79fed9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8757ea26513a41cdbba0cec7eff60ced"
I20260812 16:36:18.479000 10883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b8859d22d764e3eaa800d0cb4e5177e" candidate_uuid: "6e5848bdb7c84a8394090972ea79fed9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "588c11fd81bf448f9a31c562b9e3702e"
I20260812 16:36:18.479068 10883 raft_consensus.cc:3037] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:18.479061 10732 raft_consensus.cc:3037] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:18.480018 10883 raft_consensus.cc:2454] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e5848bdb7c84a8394090972ea79fed9 in term 1.
I20260812 16:36:18.480018 10732 raft_consensus.cc:2454] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e5848bdb7c84a8394090972ea79fed9 in term 1.
I20260812 16:36:18.480213 10494 leader_election.cc:302] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e5848bdb7c84a8394090972ea79fed9, 8757ea26513a41cdbba0cec7eff60ced; no voters:
I20260812 16:36:18.480341 10957 raft_consensus.cc:2781] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:18.480530 10957 raft_consensus.cc:686] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 [term 1 LEADER]: Becoming Leader. State: Replica: 6e5848bdb7c84a8394090972ea79fed9, State: Running, Role: LEADER
I20260812 16:36:18.480645 10957 consensus_queue.cc:237] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5848bdb7c84a8394090972ea79fed9" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43427 } } peers { permanent_uuid: "8757ea26513a41cdbba0cec7eff60ced" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41965 } } peers { permanent_uuid: "588c11fd81bf448f9a31c562b9e3702e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45297 } }
I20260812 16:36:18.481595 10405 catalog_manager.cc:5495] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e5848bdb7c84a8394090972ea79fed9 (127.19.238.65). New cstate: current_term: 1 leader_uuid: "6e5848bdb7c84a8394090972ea79fed9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5848bdb7c84a8394090972ea79fed9" member_type: VOTER last_known_addr { host: "127.19.238.65" port: 43427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8757ea26513a41cdbba0cec7eff60ced" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "588c11fd81bf448f9a31c562b9e3702e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45297 } health_report { overall_health: UNKNOWN } } }
I20260812 16:36:18.494058 10946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:18.491871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49436 (local address 127.19.238.65:43427)
0812 16:36:18.491919 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:18.491921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:18.491936 (+ 15us) server_negotiation.cc:407] Connection header received
0812 16:36:18.491971 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:18.491972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:18.491989 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:18.492050 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:36:18.492174 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.492639 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.493357 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:18.493549 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:18.493601 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:18.493731 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:36:18.493824 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:36:18.493883 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:18.493920 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 16:36:18.493937 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:36:18.495869 10732 raft_consensus.cc:1261] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced [term 1 FOLLOWER]: Refusing update from remote peer 6e5848bdb7c84a8394090972ea79fed9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:36:18.495874 10883 raft_consensus.cc:1261] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e [term 1 FOLLOWER]: Refusing update from remote peer 6e5848bdb7c84a8394090972ea79fed9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:36:18.496275 10957 consensus_queue.cc:1035] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8757ea26513a41cdbba0cec7eff60ced" member_type: VOTER last_known_addr { host: "127.19.238.66" port: 41965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:36:18.496411 10960 consensus_queue.cc:1035] T 7b8859d22d764e3eaa800d0cb4e5177e P 6e5848bdb7c84a8394090972ea79fed9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "588c11fd81bf448f9a31c562b9e3702e" member_type: VOTER last_known_addr { host: "127.19.238.67" port: 45297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:36:18.499141 10964 mvcc.cc:205] Tried to move back new op lower bound from 7317719361516191744 to 7317719361457324032. Current Snapshot: MvccSnapshot[applied={T|T < 7317719361516191744}]
I20260812 16:36:18.499388 10963 mvcc.cc:205] Tried to move back new op lower bound from 7317719361516191744 to 7317719361457324032. Current Snapshot: MvccSnapshot[applied={T|T < 7317719361516191744 or (T in {7317719361516191744})}]
I20260812 16:36:18.501341 10406 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:36:18.501457 10406 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:36:18.502255 10406 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d63dc2d5f65b462d9e727f052c4743fa: Sending DeleteTablet for 3 replicas of tablet 7b8859d22d764e3eaa800d0cb4e5177e
I20260812 16:36:18.502592 10546 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b8859d22d764e3eaa800d0cb4e5177e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:36:18 UTC) from {username='slave'} at 127.0.0.1:49430
I20260812 16:36:18.502660 10859 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b8859d22d764e3eaa800d0cb4e5177e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:36:18 UTC) from {username='slave'} at 127.0.0.1:53650
I20260812 16:36:18.502842 10704 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b8859d22d764e3eaa800d0cb4e5177e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:36:18 UTC) from {username='slave'} at 127.0.0.1:35728
I20260812 16:36:18.503028 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 10475
I20260812 16:36:18.505751 10977 tablet_replica.cc:330] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced: stopping tablet replica
I20260812 16:36:18.505874 10977 raft_consensus.cc:2229] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:36:18.505988 10977 raft_consensus.cc:2258] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:36:18.506167 10976 tablet_replica.cc:330] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e: stopping tablet replica
I20260812 16:36:18.506263 10976 raft_consensus.cc:2229] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:36:18.506356 10976 raft_consensus.cc:2258] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:36:18.506619 10977 ts_tablet_manager.cc:1905] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:36:18.507309 10976 ts_tablet_manager.cc:1905] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:36:18.508520 10977 ts_tablet_manager.cc:1918] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:36:18.508596 10977 log.cc:1196] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-1/wal/wals/7b8859d22d764e3eaa800d0cb4e5177e
I20260812 16:36:18.508860 10977 ts_tablet_manager.cc:1939] T 7b8859d22d764e3eaa800d0cb4e5177e P 8757ea26513a41cdbba0cec7eff60ced: Deleting consensus metadata
I20260812 16:36:18.509066 10976 ts_tablet_manager.cc:1918] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:36:18.509143 10976 log.cc:1196] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e: Deleting WAL directory at /tmp/dist-test-taskt79009/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786552327177731-20409-0/minicluster-data/ts-2/wal/wals/7b8859d22d764e3eaa800d0cb4e5177e
I20260812 16:36:18.509299 10392 catalog_manager.cc:4841] TS 8757ea26513a41cdbba0cec7eff60ced (127.19.238.66:41965): tablet 7b8859d22d764e3eaa800d0cb4e5177e (table test-table [id=f9383cd49d7545b281079c08983d66cf]) successfully deleted
I20260812 16:36:18.509572 10976 ts_tablet_manager.cc:1939] T 7b8859d22d764e3eaa800d0cb4e5177e P 588c11fd81bf448f9a31c562b9e3702e: Deleting consensus metadata
I20260812 16:36:18.510001 10389 catalog_manager.cc:4841] TS 588c11fd81bf448f9a31c562b9e3702e (127.19.238.67:45297): tablet 7b8859d22d764e3eaa800d0cb4e5177e (table test-table [id=f9383cd49d7545b281079c08983d66cf]) successfully deleted
W20260812 16:36:18.511353 10389 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.238.65:43427 (error 108)
I20260812 16:36:18.511860 10948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:18.511695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.65:43427 (local address 127.0.0.1:49452)
0812 16:36:18.511761 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:18.511808 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.65:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 16:36:18.511891 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 10633
W20260812 16:36:18.511977 10389 catalog_manager.cc:4584] TS 6e5848bdb7c84a8394090972ea79fed9 (127.19.238.65:43427): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7b8859d22d764e3eaa800d0cb4e5177e: Network error: Client connection negotiation failed: client connection to 127.19.238.65:43427: connect: Connection refused (error 111)
I20260812 16:36:18.519703 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 10787
I20260812 16:36:18.525920 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 10373
2026-08-12T16:36:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (733 ms)
[----------] 6 tests from EncryptionPolicyTest (9004 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:18 dist-test-slave-mvvj krb5kdc[10982](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:18 dist-test-slave-mvvj krb5kdc[10982](info): set up 2 sockets
Aug 12 16:36:18 dist-test-slave-mvvj krb5kdc[10982](info): commencing operation
krb5kdc: starting...
W20260812 16:36:19.711751 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:19 dist-test-slave-mvvj krb5kdc[10982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:36:19.843823 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42069
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:19.929018 11000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:19.929229 11000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:19.929253 11000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:19.930454 11000 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:19.930500 11000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:19.930516 11000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:19.930527 11000 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:19.930539 11000 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:19.932140 11000 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42069
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:19.932404 11000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:19.932662 11000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:19.937708 11010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:19.937721 11013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:19.937690 11009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:19.937975 11000 server_base.cc:1034] running on GCE node
I20260812 16:36:19.938323 11000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:19.938585 11000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:19.939755 11000 hybrid_clock.cc:648] HybridClock initialized: now 1786552579939738 us; error 34 us; skew 500 ppm
Aug 12 16:36:19 dist-test-slave-mvvj krb5kdc[10982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552579, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:19.950460 11000 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:36:19.951049 11000 webserver.cc:460] Webserver started at http://127.0.0.1:36645/ using document root <none> and password file <none>
I20260812 16:36:19.951349 11000 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:19.951431 11000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:19.951601 11000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:19.952901 11000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "4ee718c4ec50438bb2baf133646897bd"
format_stamp: "Formatted at 2026-08-12 16:36:19 on dist-test-slave-mvvj"
server_key: "17b4ec511ad58f431552b88664afed5e"
server_key_iv: "8f0b78184b988b6ce47737b6802a51c4"
server_key_version: "encryptionkey@0"
I20260812 16:36:19.953336 11000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "4ee718c4ec50438bb2baf133646897bd"
format_stamp: "Formatted at 2026-08-12 16:36:19 on dist-test-slave-mvvj"
server_key: "17b4ec511ad58f431552b88664afed5e"
server_key_iv: "8f0b78184b988b6ce47737b6802a51c4"
server_key_version: "encryptionkey@0"
I20260812 16:36:19.954850 11000 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:36:19.955699 11021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:19.955916 11000 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:19.956017 11000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "4ee718c4ec50438bb2baf133646897bd"
format_stamp: "Formatted at 2026-08-12 16:36:19 on dist-test-slave-mvvj"
server_key: "17b4ec511ad58f431552b88664afed5e"
server_key_iv: "8f0b78184b988b6ce47737b6802a51c4"
server_key_version: "encryptionkey@0"
I20260812 16:36:19.956103 11000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:19.963547 11000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:19.965942 11000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:19.966116 11000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:19.970711 11000 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42069
I20260812 16:36:19.970741 11078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42069 every 8 connection(s)
I20260812 16:36:19.971129 11000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:19.972108 11079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:19.974571 11079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd: Bootstrap starting.
I20260812 16:36:19.975158 11079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:19.975414 11079 log.cc:824] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:19.976269 11079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd: No bootstrap required, opened a new log
I20260812 16:36:19.977954 11079 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee718c4ec50438bb2baf133646897bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42069 } }
I20260812 16:36:19.978112 11079 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:19.978137 11079 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee718c4ec50438bb2baf133646897bd, State: Initialized, Role: FOLLOWER
I20260812 16:36:19.978282 11079 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee718c4ec50438bb2baf133646897bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42069 } }
I20260812 16:36:19.978386 11079 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:19.978410 11079 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:19.978439 11079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:19.978958 11079 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee718c4ec50438bb2baf133646897bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42069 } }
I20260812 16:36:19.979053 11079 leader_election.cc:302] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ee718c4ec50438bb2baf133646897bd; no voters:
I20260812 16:36:19.979189 11079 leader_election.cc:288] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:19.979252 11083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:19.979470 11083 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd [term 1 LEADER]: Becoming Leader. State: Replica: 4ee718c4ec50438bb2baf133646897bd, State: Running, Role: LEADER
I20260812 16:36:19.979552 11079 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:19.979569 11083 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee718c4ec50438bb2baf133646897bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42069 } }
I20260812 16:36:19.980168 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 11000
I20260812 16:36:19.980233 11084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ee718c4ec50438bb2baf133646897bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee718c4ec50438bb2baf133646897bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42069 } } }
I20260812 16:36:19.980310 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:19.980335 11084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:19.980554 20409 external_mini_cluster.cc:1428] Setting key 3d9ec67b30ffa5693f7892ac4e85c774
I20260812 16:36:19.980710 11085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ee718c4ec50438bb2baf133646897bd. Latest consensus state: current_term: 1 leader_uuid: "4ee718c4ec50438bb2baf133646897bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee718c4ec50438bb2baf133646897bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42069 } } }
I20260812 16:36:19.980798 11085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:19.981115 11093 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:19.981755 11093 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:19.984258 11093 catalog_manager.cc:1347] Generated new cluster ID: b282adf889484f2ab201c604c1e86203
I20260812 16:36:19.984314 11093 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:36:19 dist-test-slave-mvvj krb5kdc[10982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 16:36:19.993462 11102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:19.981288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56879 (local address 127.0.0.1:42069)
0812 16:36:19.981455 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:19.981466 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:19.981504 (+ 38us) server_negotiation.cc:407] Connection header received
0812 16:36:19.981641 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:19.981650 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:19.981711 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:19.982694 (+ 983us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:19.982792 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:19.984189 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:19.985348 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:19.985967 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:19.986113 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:19.989116 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:19.989120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:19.989121 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:19.989134 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:19.991286 (+ 2152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:19.991766 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:19.991770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:19.991770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:19.991933 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:19.992163 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:19.992164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:19.992164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:19.992523 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:19.993035 (+ 512us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:19.993117 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 16:36:19.993222 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260812 16:36:19.994843 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:19.996065 11093 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:19.996754 11093 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:20.003341 11093 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ee718c4ec50438bb2baf133646897bd: Generated new TSK 0
I20260812 16:36:20.003563 11093 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:20.007666 11102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:20.000310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59869 (local address 127.0.0.1:42069)
0812 16:36:20.000410 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:20.000412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:20.000565 (+ 153us) server_negotiation.cc:407] Connection header received
0812 16:36:20.000663 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:20.000664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:20.000676 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:20.000755 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:20.001372 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:20.001934 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:20.002669 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:20.002902 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:20.003002 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:20.004513 (+ 1511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:20.004514 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:20.004515 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:20.004522 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:20.006138 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:20.006596 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:20.006598 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:20.006598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:20.006671 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:20.006958 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:20.006960 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:20.006960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:20.007095 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:20.007223 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:20.007439 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 16:36:20.007521 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 16:36:20.009928 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 11000
W20260812 16:36:20.017488 10980 connection.cc:535] client connection to 127.0.0.1:42069 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42069): Connection reset by peer (error 104)
2026-08-12T16:36:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1484 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:20 dist-test-slave-mvvj krb5kdc[11112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:20 dist-test-slave-mvvj krb5kdc[11112](info): set up 2 sockets
Aug 12 16:36:20 dist-test-slave-mvvj krb5kdc[11112](info): commencing operation
krb5kdc: starting...
W20260812 16:36:21.122776 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:21 dist-test-slave-mvvj krb5kdc[11112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:36:21.255810 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38307
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:21.339679 11127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:21.339946 11127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:21.339988 11127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:21.341136 11127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:21.341209 11127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:21.341234 11127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:21.341264 11127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:21.341288 11127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:21.342803 11127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38307
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:21.343091 11127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:21.343335 11127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 16:36:21.348655 11127 server_base.cc:1034] running on GCE node
W20260812 16:36:21.348665 11134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:21.348667 11138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:21.348795 11136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:21.349162 11127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:21.349521 11127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:21.350689 11127 hybrid_clock.cc:648] HybridClock initialized: now 1786552581350683 us; error 37 us; skew 500 ppm
Aug 12 16:36:21 dist-test-slave-mvvj krb5kdc[11112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552581, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:21.353456 11127 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:36:21.354005 11127 webserver.cc:460] Webserver started at http://127.0.0.1:34631/ using document root <none> and password file <none>
I20260812 16:36:21.354286 11127 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:21.354424 11127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:21.354599 11127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:21.355818 11127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "8945de8c0975497990608a6771989dd3"
format_stamp: "Formatted at 2026-08-12 16:36:21 on dist-test-slave-mvvj"
server_key: "52958019e3cb4c5175dfac47e98199a3"
server_key_iv: "b50850f51cfa3f8a132fc0e6cd7509ba"
server_key_version: "encryptionkey@0"
I20260812 16:36:21.356307 11127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "8945de8c0975497990608a6771989dd3"
format_stamp: "Formatted at 2026-08-12 16:36:21 on dist-test-slave-mvvj"
server_key: "52958019e3cb4c5175dfac47e98199a3"
server_key_iv: "b50850f51cfa3f8a132fc0e6cd7509ba"
server_key_version: "encryptionkey@0"
I20260812 16:36:21.357863 11127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:36:21.358768 11147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:21.358983 11127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:21.359102 11127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "8945de8c0975497990608a6771989dd3"
format_stamp: "Formatted at 2026-08-12 16:36:21 on dist-test-slave-mvvj"
server_key: "52958019e3cb4c5175dfac47e98199a3"
server_key_iv: "b50850f51cfa3f8a132fc0e6cd7509ba"
server_key_version: "encryptionkey@0"
I20260812 16:36:21.359197 11127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:21.379447 11127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:21.381541 11127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:21.381736 11127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:21.386189 11127 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38307
I20260812 16:36:21.386298 11207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38307 every 8 connection(s)
I20260812 16:36:21.386672 11127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:21.387737 11208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:21.390367 11208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3: Bootstrap starting.
I20260812 16:36:21.391047 11208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:21.391383 11208 log.cc:824] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:21.392045 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 11127
I20260812 16:36:21.392167 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:21.392297 11208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3: No bootstrap required, opened a new log
I20260812 16:36:21.392428 20409 external_mini_cluster.cc:1428] Setting key 78bfaa33c9e1667b5ff5866dc3abb389
I20260812 16:36:21.394145 11208 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8945de8c0975497990608a6771989dd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38307 } }
I20260812 16:36:21.394315 11208 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:21.394363 11208 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8945de8c0975497990608a6771989dd3, State: Initialized, Role: FOLLOWER
I20260812 16:36:21.394502 11208 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8945de8c0975497990608a6771989dd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38307 } }
I20260812 16:36:21.394605 11208 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:21.394649 11208 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:21.394702 11208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:21.395470 11208 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8945de8c0975497990608a6771989dd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38307 } }
I20260812 16:36:21.395584 11208 leader_election.cc:302] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8945de8c0975497990608a6771989dd3; no voters:
I20260812 16:36:21.395753 11208 leader_election.cc:288] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:21.395815 11213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:21.396013 11208 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:21.396023 11213 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 8945de8c0975497990608a6771989dd3, State: Running, Role: LEADER
I20260812 16:36:21.396144 11213 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8945de8c0975497990608a6771989dd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38307 } }
I20260812 16:36:21.396539 11215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8945de8c0975497990608a6771989dd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8945de8c0975497990608a6771989dd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38307 } } }
I20260812 16:36:21.396633 11215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:21.396800 11216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8945de8c0975497990608a6771989dd3. Latest consensus state: current_term: 1 leader_uuid: "8945de8c0975497990608a6771989dd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8945de8c0975497990608a6771989dd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38307 } } }
I20260812 16:36:21.396883 11216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:21.396993 11225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:21.397545 11225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:21.399245 11225 catalog_manager.cc:1347] Generated new cluster ID: c4aa10b2a26849eab444de1645c5fdf3
I20260812 16:36:21.399296 11225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:21.400120 11212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:21.393039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41841 (local address 127.0.0.1:38307)
0812 16:36:21.393255 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:21.393265 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:21.393314 (+ 49us) server_negotiation.cc:407] Connection header received
0812 16:36:21.394133 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:21.394140 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:21.394193 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:21.394451 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:21.394730 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:21.396545 (+ 1815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:21.397772 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:21.398481 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:21.398593 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:21.399384 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:21.399389 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:21.399389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:21.399595 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:21.399633 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:21.399648 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:21.399743 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:21.399868 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 16:36:21.399899 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":85,"threads_started":1}
I20260812 16:36:21.401101 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:21.406025 11212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:21.402591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53243 (local address 127.0.0.1:38307)
0812 16:36:21.402626 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:21.402628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:21.402652 (+ 24us) server_negotiation.cc:407] Connection header received
0812 16:36:21.402699 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:21.402700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:21.402709 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:21.402766 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:21.402890 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:21.403615 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:21.404615 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:21.404947 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:21.405036 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:21.405752 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:21.405754 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:21.405754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:21.405759 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:21.405776 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:21.405781 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:21.405840 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:21.405923 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 16:36:21.405935 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:36:21.420123 11225 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:21.420909 11225 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:21.428717 11225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8945de8c0975497990608a6771989dd3: Generated new TSK 0
I20260812 16:36:21.429023 11225 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:21.435339 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 11127
W20260812 16:36:21.443607 11110 connection.cc:535] client connection to 127.0.0.1:38307 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38307): Connection reset by peer (error 104)
2026-08-12T16:36:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1426 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:21 dist-test-slave-mvvj krb5kdc[11247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:21 dist-test-slave-mvvj krb5kdc[11247](info): set up 2 sockets
Aug 12 16:36:21 dist-test-slave-mvvj krb5kdc[11247](info): commencing operation
krb5kdc: starting...
W20260812 16:36:22.599526 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:22 dist-test-slave-mvvj krb5kdc[11247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:36:22.736476 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44457
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:22.823685 11260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:22.823985 11260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:22.824029 11260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:22.825265 11260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:22.825341 11260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:22.825366 11260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:22.825387 11260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:22.825417 11260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:22.826962 11260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44457
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:22.827260 11260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:22.827508 11260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:22.832481 11265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:22.832535 11268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:22.832719 11266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:22.832955 11260 server_base.cc:1034] running on GCE node
I20260812 16:36:22.833294 11260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:22.833603 11260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:22.834781 11260 hybrid_clock.cc:648] HybridClock initialized: now 1786552582834768 us; error 34 us; skew 500 ppm
Aug 12 16:36:22 dist-test-slave-mvvj krb5kdc[11247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:22.837683 11260 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:36:22.838186 11260 webserver.cc:460] Webserver started at http://127.0.0.1:39043/ using document root <none> and password file <none>
I20260812 16:36:22.838465 11260 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:22.838542 11260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:22.838698 11260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:22.840173 11260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "e4e666a2626b4c86b2aa24ef5b6e877e"
format_stamp: "Formatted at 2026-08-12 16:36:22 on dist-test-slave-mvvj"
server_key: "81068aacd3742cba8cae7b8323992a84"
server_key_iv: "8226e433b20f89f551ec84bd24c77caf"
server_key_version: "encryptionkey@0"
I20260812 16:36:22.840588 11260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "e4e666a2626b4c86b2aa24ef5b6e877e"
format_stamp: "Formatted at 2026-08-12 16:36:22 on dist-test-slave-mvvj"
server_key: "81068aacd3742cba8cae7b8323992a84"
server_key_iv: "8226e433b20f89f551ec84bd24c77caf"
server_key_version: "encryptionkey@0"
I20260812 16:36:22.842131 11260 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:36:22.843076 11277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:22.843309 11260 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:22.843459 11260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "e4e666a2626b4c86b2aa24ef5b6e877e"
format_stamp: "Formatted at 2026-08-12 16:36:22 on dist-test-slave-mvvj"
server_key: "81068aacd3742cba8cae7b8323992a84"
server_key_iv: "8226e433b20f89f551ec84bd24c77caf"
server_key_version: "encryptionkey@0"
I20260812 16:36:22.843550 11260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:22.847141 11260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:22.849673 11260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:22.849843 11260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:22.854085 11260 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44457
I20260812 16:36:22.854120 11337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44457 every 8 connection(s)
I20260812 16:36:22.854526 11260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:22.855912 11338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:22.858333 11338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e: Bootstrap starting.
I20260812 16:36:22.858953 11338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:22.859251 11338 log.cc:824] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:22.860119 11338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e: No bootstrap required, opened a new log
I20260812 16:36:22.861878 11338 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4e666a2626b4c86b2aa24ef5b6e877e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44457 } }
I20260812 16:36:22.862061 11338 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:22.862109 11338 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4e666a2626b4c86b2aa24ef5b6e877e, State: Initialized, Role: FOLLOWER
I20260812 16:36:22.862306 11338 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4e666a2626b4c86b2aa24ef5b6e877e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44457 } }
I20260812 16:36:22.862443 11338 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:22.862486 11338 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:22.862541 11338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:22.862635 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 11260
I20260812 16:36:22.862769 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:22.863013 20409 external_mini_cluster.cc:1428] Setting key ab2ca086f95e0690a68451a909b300ae
I20260812 16:36:22.863286 11338 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4e666a2626b4c86b2aa24ef5b6e877e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44457 } }
I20260812 16:36:22.863420 11338 leader_election.cc:302] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4e666a2626b4c86b2aa24ef5b6e877e; no voters:
I20260812 16:36:22.863912 11338 leader_election.cc:288] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:22.863986 11344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:22.864200 11344 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e [term 1 LEADER]: Becoming Leader. State: Replica: e4e666a2626b4c86b2aa24ef5b6e877e, State: Running, Role: LEADER
I20260812 16:36:22.864220 11338 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:22.864296 11344 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4e666a2626b4c86b2aa24ef5b6e877e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44457 } }
I20260812 16:36:22.865052 11345 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4e666a2626b4c86b2aa24ef5b6e877e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4e666a2626b4c86b2aa24ef5b6e877e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44457 } } }
I20260812 16:36:22.865160 11345 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:22.865348 11346 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4e666a2626b4c86b2aa24ef5b6e877e. Latest consensus state: current_term: 1 leader_uuid: "e4e666a2626b4c86b2aa24ef5b6e877e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4e666a2626b4c86b2aa24ef5b6e877e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44457 } } }
I20260812 16:36:22.865432 11346 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:22.865913 11343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:22.863583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40969 (local address 127.0.0.1:44457)
0812 16:36:22.863797 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:22.863808 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:22.863868 (+ 60us) server_negotiation.cc:407] Connection header received
0812 16:36:22.864028 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:22.864036 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:22.864077 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:22.864586 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:22.865076 (+ 490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:22.865080 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:22.865080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:22.865085 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:22.865136 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:22.865252 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:22.865565 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:22.865600 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 16:36:22.865649 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":120,"threads_started":1}
W20260812 16:36:22.866473 11363 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:36:22.866535 11363 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:36:22.866622 11356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:22.866916 11356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:22.867069 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:22.868490 11356 catalog_manager.cc:1347] Generated new cluster ID: 5c007a195ebb4836ae759ac59b0ee36f
I20260812 16:36:22.868552 11356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:22.873356 11343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:22.872150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36355 (local address 127.0.0.1:44457)
0812 16:36:22.872203 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:22.872206 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:22.872313 (+ 107us) server_negotiation.cc:407] Connection header received
0812 16:36:22.872368 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:22.872369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:22.872378 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:22.872464 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:22.873110 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:22.873112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:22.873112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:22.873116 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:22.873135 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:22.873140 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:22.873206 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:22.873262 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 16:36:22.873276 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 16:36:22.887027 11356 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:22.887235 11356 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:22.896247 11356 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4e666a2626b4c86b2aa24ef5b6e877e: Generated new TSK 0
I20260812 16:36:22.896507 11356 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:22.899227 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 11260
2026-08-12T16:36:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1462 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:22 dist-test-slave-mvvj krb5kdc[11374](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:22 dist-test-slave-mvvj krb5kdc[11374](info): set up 2 sockets
Aug 12 16:36:22 dist-test-slave-mvvj krb5kdc[11374](info): commencing operation
krb5kdc: starting...
W20260812 16:36:24.036911 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:24 dist-test-slave-mvvj krb5kdc[11374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:36:24.169034 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44693
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:24.252347 11391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:24.252593 11391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:24.252632 11391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:24.253784 11391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:24.253854 11391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:24.253878 11391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:24.253906 11391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:24.253937 11391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:24.255460 11391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:24.255755 11391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:24.256011 11391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:24.260957 11400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:24.260986 11398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:24.261050 11397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:24.261075 11391 server_base.cc:1034] running on GCE node
I20260812 16:36:24.261369 11391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:24.261682 11391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:24.262846 11391 hybrid_clock.cc:648] HybridClock initialized: now 1786552584262833 us; error 33 us; skew 500 ppm
Aug 12 16:36:24 dist-test-slave-mvvj krb5kdc[11374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552584, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:24.265570 11391 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:36:24.266103 11391 webserver.cc:460] Webserver started at http://127.0.0.1:40145/ using document root <none> and password file <none>
I20260812 16:36:24.266351 11391 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:24.266431 11391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:24.266590 11391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:24.267726 11391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "c3aaadde90ea4fcd9407512240e858a8"
format_stamp: "Formatted at 2026-08-12 16:36:24 on dist-test-slave-mvvj"
server_key: "d43c7bec341889184b87a00bf4a95a43"
server_key_iv: "88bdb3e2c7dfb0de56bac630fbaa500f"
server_key_version: "encryptionkey@0"
I20260812 16:36:24.268180 11391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "c3aaadde90ea4fcd9407512240e858a8"
format_stamp: "Formatted at 2026-08-12 16:36:24 on dist-test-slave-mvvj"
server_key: "d43c7bec341889184b87a00bf4a95a43"
server_key_iv: "88bdb3e2c7dfb0de56bac630fbaa500f"
server_key_version: "encryptionkey@0"
I20260812 16:36:24.269608 11391 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:24.270416 11407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:24.270637 11391 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:24.270748 11391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "c3aaadde90ea4fcd9407512240e858a8"
format_stamp: "Formatted at 2026-08-12 16:36:24 on dist-test-slave-mvvj"
server_key: "d43c7bec341889184b87a00bf4a95a43"
server_key_iv: "88bdb3e2c7dfb0de56bac630fbaa500f"
server_key_version: "encryptionkey@0"
I20260812 16:36:24.270828 11391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:24.294787 11391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:24.297362 11391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:24.297600 11391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:24.302080 11391 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44693
I20260812 16:36:24.302119 11465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44693 every 8 connection(s)
I20260812 16:36:24.302505 11391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:24.303419 11466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:24.304943 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 11391
I20260812 16:36:24.305078 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:24.305325 20409 external_mini_cluster.cc:1428] Setting key fe1651c61e32a33261ad8a21de837069
I20260812 16:36:24.306545 11466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8: Bootstrap starting.
I20260812 16:36:24.307276 11466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:24.307659 11466 log.cc:824] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:24.308408 11466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8: No bootstrap required, opened a new log
I20260812 16:36:24.310874 11466 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3aaadde90ea4fcd9407512240e858a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44693 } }
I20260812 16:36:24.311092 11466 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:24.311138 11466 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3aaadde90ea4fcd9407512240e858a8, State: Initialized, Role: FOLLOWER
I20260812 16:36:24.311285 11466 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3aaadde90ea4fcd9407512240e858a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44693 } }
I20260812 16:36:24.311378 11466 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:24.311415 11466 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:24.311481 11466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:24.312420 11466 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3aaadde90ea4fcd9407512240e858a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44693 } }
I20260812 16:36:24.312527 11466 leader_election.cc:302] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3aaadde90ea4fcd9407512240e858a8; no voters:
I20260812 16:36:24.312752 11466 leader_election.cc:288] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:24.312776 11472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:24.312897 11472 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8 [term 1 LEADER]: Becoming Leader. State: Replica: c3aaadde90ea4fcd9407512240e858a8, State: Running, Role: LEADER
I20260812 16:36:24.312994 11472 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3aaadde90ea4fcd9407512240e858a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44693 } }
I20260812 16:36:24.313046 11466 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8 [sys.catalog]: configured and running, proceeding with master startup.
Aug 12 16:36:24 dist-test-slave-mvvj krb5kdc[11374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 16:36:24.313318 11472 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3aaadde90ea4fcd9407512240e858a8. Latest consensus state: current_term: 1 leader_uuid: "c3aaadde90ea4fcd9407512240e858a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3aaadde90ea4fcd9407512240e858a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44693 } } }
I20260812 16:36:24.313473 11472 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:24.313364 11473 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3aaadde90ea4fcd9407512240e858a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3aaadde90ea4fcd9407512240e858a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44693 } } }
I20260812 16:36:24.313771 11473 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:24.313970 11480 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:24.314533 11480 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:24.316406 11480 catalog_manager.cc:1347] Generated new cluster ID: 0f6ea879ee7349e9a89a9d883b306be3
I20260812 16:36:24.316460 11480 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:24.318122 11470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:24.305961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58269 (local address 127.0.0.1:44693)
0812 16:36:24.306157 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:24.306164 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:24.306191 (+ 27us) server_negotiation.cc:407] Connection header received
0812 16:36:24.306726 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:24.306734 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:24.306785 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:24.306911 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:24.307809 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:24.308806 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:24.309906 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:24.310352 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:24.310453 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:24.313755 (+ 3302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:24.313760 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:24.313760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:24.313774 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:24.316172 (+ 2398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:24.316617 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:24.316620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:24.316620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:24.316719 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:24.316908 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:24.316909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:24.316909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:24.317205 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:24.317712 (+ 507us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:24.317779 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 16:36:24.317913 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":104,"threads_started":1}
I20260812 16:36:24.319052 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:24.327261 11480 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:24.328109 11480 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:24.328415 11470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:24.320700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42071 (local address 127.0.0.1:44693)
0812 16:36:24.320742 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:24.320744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:24.320778 (+ 34us) server_negotiation.cc:407] Connection header received
0812 16:36:24.320800 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:24.320800 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:24.320810 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:24.320856 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:24.321486 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:24.322300 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:24.323281 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:24.323608 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:24.323686 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:24.325082 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:24.325084 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:24.325084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:24.325092 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:24.326820 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:24.327252 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:24.327253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:24.327253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:24.327373 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:24.327591 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:24.327592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:24.327592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:24.327728 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:24.327811 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:24.328230 (+ 419us) server_negotiation.cc:299] Negotiation successful
0812 16:36:24.328321 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:36:24.333977 11480 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3aaadde90ea4fcd9407512240e858a8: Generated new TSK 0
I20260812 16:36:24.334183 11480 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:24.339923 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 11391
2026-08-12T16:36:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:24 dist-test-slave-mvvj krb5kdc[11503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:24 dist-test-slave-mvvj krb5kdc[11503](info): set up 2 sockets
Aug 12 16:36:24 dist-test-slave-mvvj krb5kdc[11503](info): commencing operation
krb5kdc: starting...
W20260812 16:36:25.494102 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:25 dist-test-slave-mvvj krb5kdc[11503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:36:25.626941 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34179
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:25.710537 11517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:25.710776 11517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:25.710801 11517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:25.712059 11517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:25.712110 11517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:25.712126 11517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:25.712138 11517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:25.712150 11517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:25.713773 11517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:25.714066 11517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:25.714336 11517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:25.719967 11524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:25.720065 11517 server_base.cc:1034] running on GCE node
W20260812 16:36:25.719961 11526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:25.719960 11523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:25.720517 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:25.720804 11517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:25.722182 11517 hybrid_clock.cc:648] HybridClock initialized: now 1786552585722163 us; error 49 us; skew 500 ppm
Aug 12 16:36:25 dist-test-slave-mvvj krb5kdc[11503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552585, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:25.732095 11517 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:36:25.732754 11517 webserver.cc:460] Webserver started at http://127.0.0.1:43931/ using document root <none> and password file <none>
I20260812 16:36:25.733083 11517 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:25.733160 11517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:25.733314 11517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:25.734885 11517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "0cc99bd945ec4d0d8b64bc74825c87a3"
format_stamp: "Formatted at 2026-08-12 16:36:25 on dist-test-slave-mvvj"
server_key: "db34bcdd45c0e256a84fa3e646f20986"
server_key_iv: "5b69f4c6049ad71e2c0eb5d6e840bdc5"
server_key_version: "encryptionkey@0"
I20260812 16:36:25.735368 11517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "0cc99bd945ec4d0d8b64bc74825c87a3"
format_stamp: "Formatted at 2026-08-12 16:36:25 on dist-test-slave-mvvj"
server_key: "db34bcdd45c0e256a84fa3e646f20986"
server_key_iv: "5b69f4c6049ad71e2c0eb5d6e840bdc5"
server_key_version: "encryptionkey@0"
I20260812 16:36:25.737182 11517 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260812 16:36:25.738263 11533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:25.738476 11517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:25.738571 11517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "0cc99bd945ec4d0d8b64bc74825c87a3"
format_stamp: "Formatted at 2026-08-12 16:36:25 on dist-test-slave-mvvj"
server_key: "db34bcdd45c0e256a84fa3e646f20986"
server_key_iv: "5b69f4c6049ad71e2c0eb5d6e840bdc5"
server_key_version: "encryptionkey@0"
I20260812 16:36:25.738663 11517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:25.759500 11517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:25.761895 11517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:25.762075 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:25.766381 11517 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34179
I20260812 16:36:25.766459 11596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34179 every 8 connection(s)
I20260812 16:36:25.766830 11517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:25.767786 11597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:25.770488 11597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3: Bootstrap starting.
I20260812 16:36:25.771152 11597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:25.771482 11597 log.cc:824] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:25.772328 11597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3: No bootstrap required, opened a new log
I20260812 16:36:25.773075 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 11517
I20260812 16:36:25.773195 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:25.773404 20409 external_mini_cluster.cc:1428] Setting key f11e96f76feac87c826589cc6cd823ac
I20260812 16:36:25.774796 11597 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc99bd945ec4d0d8b64bc74825c87a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34179 } }
I20260812 16:36:25.774960 11597 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:25.774988 11597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cc99bd945ec4d0d8b64bc74825c87a3, State: Initialized, Role: FOLLOWER
I20260812 16:36:25.775177 11597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc99bd945ec4d0d8b64bc74825c87a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34179 } }
I20260812 16:36:25.775285 11597 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:25.775338 11597 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:25.775422 11597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:25.776095 11597 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc99bd945ec4d0d8b64bc74825c87a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34179 } }
I20260812 16:36:25.776225 11597 leader_election.cc:302] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cc99bd945ec4d0d8b64bc74825c87a3; no voters:
I20260812 16:36:25.776499 11597 leader_election.cc:288] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:25.776526 11604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:25.776715 11604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3 [term 1 LEADER]: Becoming Leader. State: Replica: 0cc99bd945ec4d0d8b64bc74825c87a3, State: Running, Role: LEADER
I20260812 16:36:25.776854 11597 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:25.776870 11604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc99bd945ec4d0d8b64bc74825c87a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34179 } }
I20260812 16:36:25.777388 11606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cc99bd945ec4d0d8b64bc74825c87a3. Latest consensus state: current_term: 1 leader_uuid: "0cc99bd945ec4d0d8b64bc74825c87a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc99bd945ec4d0d8b64bc74825c87a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34179 } } }
I20260812 16:36:25.777488 11606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:25.777709 11605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cc99bd945ec4d0d8b64bc74825c87a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc99bd945ec4d0d8b64bc74825c87a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34179 } } }
I20260812 16:36:25.778050 11605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:25.778254 11611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:25.778883 11611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:25.780064 11603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:25.773970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33551 (local address 127.0.0.1:34179)
0812 16:36:25.774148 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:25.774156 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:25.774193 (+ 37us) server_negotiation.cc:407] Connection header received
0812 16:36:25.774895 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:25.774903 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:25.774982 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:25.775159 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:25.775396 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:25.776489 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:25.777502 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:25.778155 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:25.778243 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:25.779242 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:25.779246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:25.779246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:25.779474 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:25.779536 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:25.779553 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:25.779632 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:25.779723 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 16:36:25.779798 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260812 16:36:25.780820 11611 catalog_manager.cc:1347] Generated new cluster ID: 796dd138cc6c409083720124271449e6
I20260812 16:36:25.780870 11611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:25.781052 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:25.786087 11611 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:25.786243 11603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:25.782532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43861 (local address 127.0.0.1:34179)
0812 16:36:25.782569 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:25.782571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:25.782689 (+ 118us) server_negotiation.cc:407] Connection header received
0812 16:36:25.782735 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:25.782735 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:25.782745 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:25.782815 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:25.782942 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:25.783648 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:25.784659 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:25.784981 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:25.785051 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:25.785960 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:25.785961 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:25.785961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:25.785967 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:25.785983 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:25.785988 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:25.786047 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:25.786159 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 16:36:25.786181 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:36:25.786790 11611 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:25.792135 11611 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cc99bd945ec4d0d8b64bc74825c87a3: Generated new TSK 0
I20260812 16:36:25.792325 11611 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:25.797732 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 11517
2026-08-12T16:36:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1459 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:25 dist-test-slave-mvvj krb5kdc[11631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:25 dist-test-slave-mvvj krb5kdc[11631](info): set up 2 sockets
Aug 12 16:36:25 dist-test-slave-mvvj krb5kdc[11631](info): commencing operation
krb5kdc: starting...
W20260812 16:36:26.919569 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:27 dist-test-slave-mvvj krb5kdc[11631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:36:27.055086 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34459
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:27.146437 11649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:27.146682 11649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:27.146723 11649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:27.147994 11649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:27.148061 11649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:27.148087 11649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:27.148119 11649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:27.148150 11649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:27.149672 11649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:27.149958 11649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:27.150195 11649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:27.155207 11655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:27.155236 11654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:27.155207 11657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:27.155470 11649 server_base.cc:1034] running on GCE node
I20260812 16:36:27.155771 11649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:27.156112 11649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:27.157263 11649 hybrid_clock.cc:648] HybridClock initialized: now 1786552587157239 us; error 37 us; skew 500 ppm
Aug 12 16:36:27 dist-test-slave-mvvj krb5kdc[11631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552587, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:27.159941 11649 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:36:27.160439 11649 webserver.cc:460] Webserver started at http://127.0.0.1:36517/ using document root <none> and password file <none>
I20260812 16:36:27.160681 11649 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:27.160724 11649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:27.160818 11649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:27.161919 11649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "88010377da73486c9e98bf1b762eb444"
format_stamp: "Formatted at 2026-08-12 16:36:27 on dist-test-slave-mvvj"
server_key: "d335eff750ff96267a912139375de8ab"
server_key_iv: "97bb2b49781fd65ea6da7b047b6e9232"
server_key_version: "encryptionkey@0"
I20260812 16:36:27.162317 11649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "88010377da73486c9e98bf1b762eb444"
format_stamp: "Formatted at 2026-08-12 16:36:27 on dist-test-slave-mvvj"
server_key: "d335eff750ff96267a912139375de8ab"
server_key_iv: "97bb2b49781fd65ea6da7b047b6e9232"
server_key_version: "encryptionkey@0"
I20260812 16:36:27.163995 11649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:36:27.164886 11665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:27.165098 11649 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:27.165207 11649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "88010377da73486c9e98bf1b762eb444"
format_stamp: "Formatted at 2026-08-12 16:36:27 on dist-test-slave-mvvj"
server_key: "d335eff750ff96267a912139375de8ab"
server_key_iv: "97bb2b49781fd65ea6da7b047b6e9232"
server_key_version: "encryptionkey@0"
I20260812 16:36:27.165270 11649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:27.168738 11649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:27.170719 11649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:27.170831 11649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:27.175920 11649 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34459
I20260812 16:36:27.175990 11733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34459 every 8 connection(s)
I20260812 16:36:27.176440 11649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:27.177584 11734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:27.181088 11734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444: Bootstrap starting.
I20260812 16:36:27.181318 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 11649
I20260812 16:36:27.181443 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:27.181689 20409 external_mini_cluster.cc:1428] Setting key f91fc5dd7ad5bc0c50bb0b131d77c281
I20260812 16:36:27.181988 11734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:27.182391 11734 log.cc:824] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:27.183353 11734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444: No bootstrap required, opened a new log
I20260812 16:36:27.184559 11737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:27.182445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34391 (local address 127.0.0.1:34459)
0812 16:36:27.182636 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:27.182644 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:27.182669 (+ 25us) server_negotiation.cc:407] Connection header received
0812 16:36:27.183147 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:27.183154 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:27.183182 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:27.183340 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:27.184146 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:27.184149 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:27.184150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:27.184154 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:27.184206 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:27.184216 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:27.184311 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:27.184350 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 16:36:27.184391 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":79,"threads_started":1}
I20260812 16:36:27.185516 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:27.185997 11734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88010377da73486c9e98bf1b762eb444" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34459 } }
I20260812 16:36:27.186156 11734 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:27.186204 11734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88010377da73486c9e98bf1b762eb444, State: Initialized, Role: FOLLOWER
I20260812 16:36:27.186334 11734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88010377da73486c9e98bf1b762eb444" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34459 } }
I20260812 16:36:27.186444 11734 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:27.186515 11734 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:27.186573 11734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:27.187165 11734 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88010377da73486c9e98bf1b762eb444" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34459 } }
I20260812 16:36:27.187258 11734 leader_election.cc:302] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88010377da73486c9e98bf1b762eb444; no voters:
I20260812 16:36:27.187398 11734 leader_election.cc:288] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:27.187465 11743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:27.187639 11743 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444 [term 1 LEADER]: Becoming Leader. State: Replica: 88010377da73486c9e98bf1b762eb444, State: Running, Role: LEADER
I20260812 16:36:27.187670 11734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:27.187788 11743 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88010377da73486c9e98bf1b762eb444" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34459 } }
I20260812 16:36:27.188766 11745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88010377da73486c9e98bf1b762eb444. Latest consensus state: current_term: 1 leader_uuid: "88010377da73486c9e98bf1b762eb444" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88010377da73486c9e98bf1b762eb444" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34459 } } }
I20260812 16:36:27.188886 11745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:27.189100 11744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88010377da73486c9e98bf1b762eb444" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88010377da73486c9e98bf1b762eb444" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34459 } } }
I20260812 16:36:27.189208 11744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:27.189087 11759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:27.189885 11737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:27.187947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47149 (local address 127.0.0.1:34459)
0812 16:36:27.187986 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:27.187988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:27.188244 (+ 256us) server_negotiation.cc:407] Connection header received
0812 16:36:27.188312 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:27.188312 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:27.188317 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:27.188380 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:27.189633 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:27.189634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:27.189635 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:27.189639 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:27.189654 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:27.189659 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:27.189714 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:27.189793 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 16:36:27.189806 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:36:27.190158 11759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:27.191659 11759 catalog_manager.cc:1347] Generated new cluster ID: d17795e624d24416ad1697d8e79ac079
I20260812 16:36:27.191712 11759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:27.220755 11759 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:27.221011 11759 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:27.227198 11759 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88010377da73486c9e98bf1b762eb444: Generated new TSK 0
I20260812 16:36:27.227413 11759 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:27.233140 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 11649
2026-08-12T16:36:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1433 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:27 dist-test-slave-mvvj krb5kdc[11769](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:27 dist-test-slave-mvvj krb5kdc[11769](info): set up 2 sockets
Aug 12 16:36:27 dist-test-slave-mvvj krb5kdc[11769](info): commencing operation
krb5kdc: starting...
W20260812 16:36:28.364089 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:28 dist-test-slave-mvvj krb5kdc[11769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:36:28.497213 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45551
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:28.580590 11783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:28.580808 11783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:28.580832 11783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:28.582010 11783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:28.582054 11783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:28.582069 11783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:28.582082 11783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:28.582135 11783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:28.583676 11783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:28.584003 11783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:28.584259 11783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:28.589574 11792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:28.589589 11788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:28.589574 11789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:28.589875 11783 server_base.cc:1034] running on GCE node
I20260812 16:36:28.590174 11783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:28.590508 11783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:28.591688 11783 hybrid_clock.cc:648] HybridClock initialized: now 1786552588591673 us; error 50 us; skew 500 ppm
Aug 12 16:36:28 dist-test-slave-mvvj krb5kdc[11769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552588, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:28.595152 11783 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:36:28.595654 11783 webserver.cc:460] Webserver started at http://127.0.0.1:34623/ using document root <none> and password file <none>
I20260812 16:36:28.595966 11783 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:28.596041 11783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:28.596210 11783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:28.597344 11783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "bdfd96f184a54417ad735209fe2ed632"
format_stamp: "Formatted at 2026-08-12 16:36:28 on dist-test-slave-mvvj"
server_key: "198b824a9c0357e9916908fa58fbf15a"
server_key_iv: "94b86490519a04778f5c2bbe41c72599"
server_key_version: "encryptionkey@0"
I20260812 16:36:28.597759 11783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "bdfd96f184a54417ad735209fe2ed632"
format_stamp: "Formatted at 2026-08-12 16:36:28 on dist-test-slave-mvvj"
server_key: "198b824a9c0357e9916908fa58fbf15a"
server_key_iv: "94b86490519a04778f5c2bbe41c72599"
server_key_version: "encryptionkey@0"
I20260812 16:36:28.599292 11783 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:36:28.600287 11801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:28.600490 11783 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:36:28.600605 11783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "bdfd96f184a54417ad735209fe2ed632"
format_stamp: "Formatted at 2026-08-12 16:36:28 on dist-test-slave-mvvj"
server_key: "198b824a9c0357e9916908fa58fbf15a"
server_key_iv: "94b86490519a04778f5c2bbe41c72599"
server_key_version: "encryptionkey@0"
I20260812 16:36:28.600688 11783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:28.625022 11783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:28.627164 11783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:28.627345 11783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:28.631953 11783 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45551
I20260812 16:36:28.631994 11863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45551 every 8 connection(s)
I20260812 16:36:28.632398 11783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:28.632850 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 11783
I20260812 16:36:28.633001 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:28.633239 20409 external_mini_cluster.cc:1428] Setting key 33a1a860b6297dc3bb4322d072d1db70
I20260812 16:36:28.633598 11864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:28.637179 11864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632: Bootstrap starting.
I20260812 16:36:28.637842 11864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:28.638192 11864 log.cc:824] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:28.639055 11864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632: No bootstrap required, opened a new log
Aug 12 16:36:28 dist-test-slave-mvvj krb5kdc[11769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 16:36:28.641201 11864 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdfd96f184a54417ad735209fe2ed632" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45551 } }
I20260812 16:36:28.641394 11864 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:28.641453 11864 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdfd96f184a54417ad735209fe2ed632, State: Initialized, Role: FOLLOWER
I20260812 16:36:28.641609 11864 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdfd96f184a54417ad735209fe2ed632" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45551 } }
I20260812 16:36:28.641721 11864 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:28.641780 11864 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:28.641832 11864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:28.642632 11864 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdfd96f184a54417ad735209fe2ed632" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45551 } }
I20260812 16:36:28.642752 11864 leader_election.cc:302] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdfd96f184a54417ad735209fe2ed632; no voters:
I20260812 16:36:28.642933 11864 leader_election.cc:288] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:28.643020 11870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:28.643199 11870 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632 [term 1 LEADER]: Becoming Leader. State: Replica: bdfd96f184a54417ad735209fe2ed632, State: Running, Role: LEADER
I20260812 16:36:28.643213 11864 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:28.643369 11870 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdfd96f184a54417ad735209fe2ed632" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45551 } }
I20260812 16:36:28.643970 11872 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdfd96f184a54417ad735209fe2ed632" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdfd96f184a54417ad735209fe2ed632" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45551 } } }
I20260812 16:36:28.644150 11872 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:28.644989 11881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:28.645241 11873 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdfd96f184a54417ad735209fe2ed632. Latest consensus state: current_term: 1 leader_uuid: "bdfd96f184a54417ad735209fe2ed632" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdfd96f184a54417ad735209fe2ed632" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45551 } } }
I20260812 16:36:28.645382 11873 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:28.645533 11881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:28.645795 11867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:28.633871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:36959 (local address 127.0.0.1:45551)
0812 16:36:28.634118 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:28.634127 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:28.634164 (+ 37us) server_negotiation.cc:407] Connection header received
0812 16:36:28.634849 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:28.634860 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:28.634932 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:28.635107 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:28.635733 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:28.637189 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:28.638080 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:28.638647 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:28.638787 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:28.641372 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:28.641376 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:28.641376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:28.641390 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:28.643514 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:28.644015 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:28.644018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:28.644019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:28.644137 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:28.644425 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:28.644426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:28.644426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:28.644736 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:28.645297 (+ 561us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:28.645401 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 16:36:28.645520 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":106,"threads_started":1}
I20260812 16:36:28.647240 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:28.648835 11881 catalog_manager.cc:1347] Generated new cluster ID: d5c809cfde214808a55815c5f482299c
I20260812 16:36:28.648892 11881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:28.655288 11867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:28.649111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:52321 (local address 127.0.0.1:45551)
0812 16:36:28.649146 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:28.649148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:28.649156 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:36:28.649179 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:28.649180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:28.649188 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:28.649233 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:28.649781 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:28.650263 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:28.651000 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:28.651229 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:28.651319 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:28.652573 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:28.652575 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:28.652575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:28.652582 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:28.653998 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:28.654450 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:28.654451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:28.654451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:28.654513 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:28.654748 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:28.654749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:28.654750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:28.654866 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:28.654985 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:28.655112 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 16:36:28.655179 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:36:28.675410 11881 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:28.676074 11881 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:28.679986 11881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdfd96f184a54417ad735209fe2ed632: Generated new TSK 0
I20260812 16:36:28.680207 11881 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:28.691725 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 11783
W20260812 16:36:28.699915 11765 connection.cc:535] client connection to 127.0.0.1:45551 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45551): Connection reset by peer (error 104)
2026-08-12T16:36:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1458 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:28 dist-test-slave-mvvj krb5kdc[11902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:28 dist-test-slave-mvvj krb5kdc[11902](info): set up 2 sockets
Aug 12 16:36:28 dist-test-slave-mvvj krb5kdc[11902](info): commencing operation
krb5kdc: starting...
W20260812 16:36:29.807458 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:29 dist-test-slave-mvvj krb5kdc[11902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:36:29.936717 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38719
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:30.020725 11917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:30.020952 11917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:30.020977 11917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:30.022095 11917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:30.022133 11917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:30.022147 11917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:30.022161 11917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:30.022172 11917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:30.023604 11917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:30.023808 11917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:30.024076 11917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:30.029168 11924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:30.029161 11923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:30.029161 11926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:30.029655 11917 server_base.cc:1034] running on GCE node
I20260812 16:36:30.029935 11917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:30.030267 11917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:30.031486 11917 hybrid_clock.cc:648] HybridClock initialized: now 1786552590031470 us; error 44 us; skew 500 ppm
Aug 12 16:36:30 dist-test-slave-mvvj krb5kdc[11902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552590, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:30.034866 11917 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:36:30.035478 11917 webserver.cc:460] Webserver started at http://127.0.0.1:40835/ using document root <none> and password file <none>
I20260812 16:36:30.035735 11917 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:30.035826 11917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:30.035992 11917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:30.037089 11917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "442d1d13db0a437d8ae1b2472e2bb247"
format_stamp: "Formatted at 2026-08-12 16:36:30 on dist-test-slave-mvvj"
server_key: "738795616c202823bc3ca57774610325"
server_key_iv: "746b740d34c116ac5c61617237def57e"
server_key_version: "encryptionkey@0"
I20260812 16:36:30.037472 11917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "442d1d13db0a437d8ae1b2472e2bb247"
format_stamp: "Formatted at 2026-08-12 16:36:30 on dist-test-slave-mvvj"
server_key: "738795616c202823bc3ca57774610325"
server_key_iv: "746b740d34c116ac5c61617237def57e"
server_key_version: "encryptionkey@0"
I20260812 16:36:30.038902 11917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:36:30.039647 11933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:30.039822 11917 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 16:36:30.039958 11917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "442d1d13db0a437d8ae1b2472e2bb247"
format_stamp: "Formatted at 2026-08-12 16:36:30 on dist-test-slave-mvvj"
server_key: "738795616c202823bc3ca57774610325"
server_key_iv: "746b740d34c116ac5c61617237def57e"
server_key_version: "encryptionkey@0"
I20260812 16:36:30.040053 11917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:30.066951 11917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:30.069263 11917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:30.069460 11917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:30.074156 11917 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38719
I20260812 16:36:30.074199 11995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38719 every 8 connection(s)
I20260812 16:36:30.074620 11917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:30.075543 11996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:30.077956 11996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247: Bootstrap starting.
I20260812 16:36:30.078596 11996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:30.078896 11996 log.cc:824] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:30.079686 11996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247: No bootstrap required, opened a new log
I20260812 16:36:30.081609 11996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "442d1d13db0a437d8ae1b2472e2bb247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38719 } }
I20260812 16:36:30.081777 11996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:30.081804 11996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442d1d13db0a437d8ae1b2472e2bb247, State: Initialized, Role: FOLLOWER
I20260812 16:36:30.082000 11996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "442d1d13db0a437d8ae1b2472e2bb247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38719 } }
I20260812 16:36:30.082080 11996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:30.082127 11996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:30.082190 11996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:30.082816 11996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "442d1d13db0a437d8ae1b2472e2bb247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38719 } }
I20260812 16:36:30.082917 11996 leader_election.cc:302] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 442d1d13db0a437d8ae1b2472e2bb247; no voters:
I20260812 16:36:30.083117 11996 leader_election.cc:288] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:30.083220 11999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:30.083417 11999 raft_consensus.cc:686] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247 [term 1 LEADER]: Becoming Leader. State: Replica: 442d1d13db0a437d8ae1b2472e2bb247, State: Running, Role: LEADER
I20260812 16:36:30.083678 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 11917
I20260812 16:36:30.083823 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:30.084059 20409 external_mini_cluster.cc:1428] Setting key 59adbf4b460a020996168f5d5e4b290f
I20260812 16:36:30.083560 11996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:30.084311 11999 consensus_queue.cc:237] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "442d1d13db0a437d8ae1b2472e2bb247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38719 } }
I20260812 16:36:30.085376 12001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 442d1d13db0a437d8ae1b2472e2bb247. Latest consensus state: current_term: 1 leader_uuid: "442d1d13db0a437d8ae1b2472e2bb247" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "442d1d13db0a437d8ae1b2472e2bb247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38719 } } }
I20260812 16:36:30.085485 12001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:30.085718 12016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:30.085749 12000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "442d1d13db0a437d8ae1b2472e2bb247" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "442d1d13db0a437d8ae1b2472e2bb247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38719 } } }
I20260812 16:36:30.086004 12000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:30.086376 12016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:30.088225 12016 catalog_manager.cc:1347] Generated new cluster ID: fe6079b236e84f69b43c4a3598f4a979
I20260812 16:36:30.088290 12016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:30.091533 12009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:30.084994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:52179 (local address 127.0.0.1:38719)
0812 16:36:30.085370 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:30.085382 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:30.086222 (+ 840us) server_negotiation.cc:407] Connection header received
0812 16:36:30.086353 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:30.086361 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:30.086439 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:30.086590 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:30.086822 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:30.088063 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:30.089481 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:30.089939 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:30.090070 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:30.090858 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:30.090862 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:30.090863 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:30.091066 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:30.091099 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:30.091111 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:30.091216 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:30.091297 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 16:36:30.091328 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260812 16:36:30.092612 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:30.094765 12016 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:30.095250 12016 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:30.098408 12009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:30.094567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:36259 (local address 127.0.0.1:38719)
0812 16:36:30.094601 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:30.094602 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:30.094723 (+ 121us) server_negotiation.cc:407] Connection header received
0812 16:36:30.094830 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:30.094831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:30.094845 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:30.094880 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:30.095115 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:30.095881 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:30.096935 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:30.097218 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:30.097260 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:30.098091 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:30.098092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:30.098093 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:30.098097 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:30.098114 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:30.098118 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:30.098151 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:30.098284 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 16:36:30.098299 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 16:36:30.099704 12016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 442d1d13db0a437d8ae1b2472e2bb247: Generated new TSK 0
I20260812 16:36:30.099928 12016 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:30.104651 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 11917
W20260812 16:36:30.111505 11900 connection.cc:535] client connection to 127.0.0.1:38719 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38719): Connection reset by peer (error 104)
2026-08-12T16:36:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1411 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:30 dist-test-slave-mvvj krb5kdc[12031](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:30 dist-test-slave-mvvj krb5kdc[12031](info): set up 2 sockets
Aug 12 16:36:30 dist-test-slave-mvvj krb5kdc[12031](info): commencing operation
krb5kdc: starting...
W20260812 16:36:31.215899 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:31 dist-test-slave-mvvj krb5kdc[12031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:36:31.342311 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43601
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:31.425490 12045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:31.425722 12045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:31.425746 12045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:31.426858 12045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:31.426896 12045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:31.426910 12045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:31.426923 12045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:31.426934 12045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:31.428475 12045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:31.428748 12045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:31.428982 12045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:31.434002 12053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:31.434062 12050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:31.434048 12051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:31.434288 12045 server_base.cc:1034] running on GCE node
I20260812 16:36:31.434528 12045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:31.434806 12045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:31.435906 12045 hybrid_clock.cc:648] HybridClock initialized: now 1786552591435885 us; error 36 us; skew 500 ppm
Aug 12 16:36:31 dist-test-slave-mvvj krb5kdc[12031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552591, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:31.443295 12045 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:36:31.443794 12045 webserver.cc:460] Webserver started at http://127.0.0.1:44019/ using document root <none> and password file <none>
I20260812 16:36:31.444082 12045 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:31.444130 12045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:31.444294 12045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:31.445365 12045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "ae0433857f434cd6a061b874bfcced09"
format_stamp: "Formatted at 2026-08-12 16:36:31 on dist-test-slave-mvvj"
server_key: "b41648e3e5ab9aa84b0be3f283dbca34"
server_key_iv: "b8d7be34e22e84672d2ac18947627728"
server_key_version: "encryptionkey@0"
I20260812 16:36:31.445729 12045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "ae0433857f434cd6a061b874bfcced09"
format_stamp: "Formatted at 2026-08-12 16:36:31 on dist-test-slave-mvvj"
server_key: "b41648e3e5ab9aa84b0be3f283dbca34"
server_key_iv: "b8d7be34e22e84672d2ac18947627728"
server_key_version: "encryptionkey@0"
I20260812 16:36:31.447158 12045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:36:31.447986 12063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:31.448189 12045 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:31.448295 12045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "ae0433857f434cd6a061b874bfcced09"
format_stamp: "Formatted at 2026-08-12 16:36:31 on dist-test-slave-mvvj"
server_key: "b41648e3e5ab9aa84b0be3f283dbca34"
server_key_iv: "b8d7be34e22e84672d2ac18947627728"
server_key_version: "encryptionkey@0"
I20260812 16:36:31.448381 12045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:31.451676 12045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:31.453644 12045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:31.453810 12045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:31.459121 12045 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43601
I20260812 16:36:31.459174 12124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43601 every 8 connection(s)
I20260812 16:36:31.459613 12045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:31.461329 12125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:31.464716 12125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09: Bootstrap starting.
I20260812 16:36:31.465426 12125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:31.465732 12125 log.cc:824] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:31.466534 12125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09: No bootstrap required, opened a new log
I20260812 16:36:31.467672 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 12045
I20260812 16:36:31.467765 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:31.467993 20409 external_mini_cluster.cc:1428] Setting key 9e3c62c9cf81b0826121c9d8a9f1e01e
I20260812 16:36:31.468771 12125 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae0433857f434cd6a061b874bfcced09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43601 } }
I20260812 16:36:31.468972 12125 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:31.469009 12125 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0433857f434cd6a061b874bfcced09, State: Initialized, Role: FOLLOWER
I20260812 16:36:31.469167 12125 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae0433857f434cd6a061b874bfcced09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43601 } }
I20260812 16:36:31.469259 12125 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:31.469357 12125 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:31.469436 12125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:31.470516 12129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:31.468610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:54473 (local address 127.0.0.1:43601)
0812 16:36:31.468839 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:31.468848 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:31.468884 (+ 36us) server_negotiation.cc:407] Connection header received
0812 16:36:31.469002 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:31.469010 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:31.469043 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:31.469190 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:31.469938 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:31.469943 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:31.469943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:31.469949 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:31.470009 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:31.470027 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:31.470119 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:31.470256 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 16:36:31.470306 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":113,"threads_started":1}
I20260812 16:36:31.470712 12125 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae0433857f434cd6a061b874bfcced09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43601 } }
I20260812 16:36:31.470830 12125 leader_election.cc:302] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae0433857f434cd6a061b874bfcced09; no voters:
I20260812 16:36:31.470997 12125 leader_election.cc:288] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:31.471043 12130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:31.471160 12130 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09 [term 1 LEADER]: Becoming Leader. State: Replica: ae0433857f434cd6a061b874bfcced09, State: Running, Role: LEADER
I20260812 16:36:31.471280 12125 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:31.471299 12130 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae0433857f434cd6a061b874bfcced09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43601 } }
I20260812 16:36:31.471519 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:31.471750 12130 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae0433857f434cd6a061b874bfcced09. Latest consensus state: current_term: 1 leader_uuid: "ae0433857f434cd6a061b874bfcced09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae0433857f434cd6a061b874bfcced09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43601 } } }
I20260812 16:36:31.471792 12131 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae0433857f434cd6a061b874bfcced09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae0433857f434cd6a061b874bfcced09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43601 } } }
I20260812 16:36:31.471863 12130 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:31.471889 12131 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:31.473070 12135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:31.473757 12135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:31.474391 12129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:31.473180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:57785 (local address 127.0.0.1:43601)
0812 16:36:31.473220 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:31.473222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:31.473328 (+ 106us) server_negotiation.cc:407] Connection header received
0812 16:36:31.473408 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:31.473409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:31.473414 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:31.473444 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:31.474153 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:31.474155 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:31.474156 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:31.474161 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:31.474174 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:31.474179 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:31.474210 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:31.474318 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 16:36:31.474332 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:36:31.475463 12135 catalog_manager.cc:1347] Generated new cluster ID: 1cf5880acf424063988fe0a79a745b47
I20260812 16:36:31.475520 12135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:31.504290 12135 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:31.504526 12135 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:31.507822 12135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae0433857f434cd6a061b874bfcced09: Generated new TSK 0
I20260812 16:36:31.508061 12135 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:31.516069 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 12045
2026-08-12T16:36:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:31 dist-test-slave-mvvj krb5kdc[12157](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:31 dist-test-slave-mvvj krb5kdc[12157](info): set up 2 sockets
Aug 12 16:36:31 dist-test-slave-mvvj krb5kdc[12157](info): commencing operation
krb5kdc: starting...
W20260812 16:36:32.628074 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:32 dist-test-slave-mvvj krb5kdc[12157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:36:32.766692 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40001
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:32.851699 12173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:32.851999 12173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:32.852042 12173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:32.853255 12173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:32.853334 12173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:32.853372 12173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:32.853415 12173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:32.853447 12173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:32.854979 12173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40001
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:32.855288 12173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:32.855535 12173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:32.860874 12181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:32.860857 12186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:32.860877 12184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:32.860973 12173 server_base.cc:1034] running on GCE node
I20260812 16:36:32.861363 12173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:32.861732 12173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:32.862920 12173 hybrid_clock.cc:648] HybridClock initialized: now 1786552592862919 us; error 65 us; skew 500 ppm
Aug 12 16:36:32 dist-test-slave-mvvj krb5kdc[12157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552592, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:32.865694 12173 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:36:32.866197 12173 webserver.cc:460] Webserver started at http://127.0.0.1:39037/ using document root <none> and password file <none>
I20260812 16:36:32.866441 12173 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:32.866513 12173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:32.866669 12173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:32.867877 12173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "169d7781f62f4bd58ff5ae6f556ba9f4"
format_stamp: "Formatted at 2026-08-12 16:36:32 on dist-test-slave-mvvj"
server_key: "43e108b2f8384133d7a96bab07fdbfd2"
server_key_iv: "d0ebd9a5879fb04fec8f083431cff6d2"
server_key_version: "encryptionkey@0"
I20260812 16:36:32.868289 12173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "169d7781f62f4bd58ff5ae6f556ba9f4"
format_stamp: "Formatted at 2026-08-12 16:36:32 on dist-test-slave-mvvj"
server_key: "43e108b2f8384133d7a96bab07fdbfd2"
server_key_iv: "d0ebd9a5879fb04fec8f083431cff6d2"
server_key_version: "encryptionkey@0"
I20260812 16:36:32.869702 12173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:36:32.870581 12194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:32.870775 12173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:36:32.870879 12173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "169d7781f62f4bd58ff5ae6f556ba9f4"
format_stamp: "Formatted at 2026-08-12 16:36:32 on dist-test-slave-mvvj"
server_key: "43e108b2f8384133d7a96bab07fdbfd2"
server_key_iv: "d0ebd9a5879fb04fec8f083431cff6d2"
server_key_version: "encryptionkey@0"
I20260812 16:36:32.870966 12173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:32.887975 12173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:32.890442 12173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:32.890617 12173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:32.895130 12173 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40001
I20260812 16:36:32.895166 12251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40001 every 8 connection(s)
I20260812 16:36:32.895576 12173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:32.896518 12252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:32.898900 12252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4: Bootstrap starting.
I20260812 16:36:32.899533 12252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:32.899879 12252 log.cc:824] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:32.900637 12252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4: No bootstrap required, opened a new log
I20260812 16:36:32.902283 12252 raft_consensus.cc:348] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "169d7781f62f4bd58ff5ae6f556ba9f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40001 } }
I20260812 16:36:32.902427 12252 raft_consensus.cc:374] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:32.902489 12252 raft_consensus.cc:729] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 169d7781f62f4bd58ff5ae6f556ba9f4, State: Initialized, Role: FOLLOWER
I20260812 16:36:32.902673 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 12173
I20260812 16:36:32.902658 12252 consensus_queue.cc:260] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "169d7781f62f4bd58ff5ae6f556ba9f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40001 } }
I20260812 16:36:32.902740 12252 raft_consensus.cc:388] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:32.902778 12252 raft_consensus.cc:482] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:32.902801 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:32.902832 12252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:32.903023 20409 external_mini_cluster.cc:1428] Setting key 69cb2298d2126b19fd8341812dd795f8
I20260812 16:36:32.903530 12252 raft_consensus.cc:504] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "169d7781f62f4bd58ff5ae6f556ba9f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40001 } }
I20260812 16:36:32.903667 12252 leader_election.cc:302] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 169d7781f62f4bd58ff5ae6f556ba9f4; no voters:
I20260812 16:36:32.903867 12252 leader_election.cc:288] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:32.903884 12259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:32.904006 12259 raft_consensus.cc:686] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4 [term 1 LEADER]: Becoming Leader. State: Replica: 169d7781f62f4bd58ff5ae6f556ba9f4, State: Running, Role: LEADER
I20260812 16:36:32.904162 12252 sys_catalog.cc:564] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:32.904150 12259 consensus_queue.cc:237] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "169d7781f62f4bd58ff5ae6f556ba9f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40001 } }
I20260812 16:36:32.904834 12260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 169d7781f62f4bd58ff5ae6f556ba9f4. Latest consensus state: current_term: 1 leader_uuid: "169d7781f62f4bd58ff5ae6f556ba9f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "169d7781f62f4bd58ff5ae6f556ba9f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40001 } } }
I20260812 16:36:32.904932 12260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:32.905313 12261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "169d7781f62f4bd58ff5ae6f556ba9f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "169d7781f62f4bd58ff5ae6f556ba9f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40001 } } }
I20260812 16:36:32.905428 12261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:32.905577 12269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:32.906272 12269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:32.908365 12269 catalog_manager.cc:1347] Generated new cluster ID: f846248be9c04d64ad04b8f421be4987
I20260812 16:36:32.908444 12269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:36:32 dist-test-slave-mvvj krb5kdc[12157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 16:36:32.914336 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:32.903558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:50333 (local address 127.0.0.1:40001)
0812 16:36:32.903767 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:32.903777 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:32.903811 (+ 34us) server_negotiation.cc:407] Connection header received
0812 16:36:32.903990 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:32.903997 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:32.904074 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:32.905051 (+ 977us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:32.905215 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:32.906479 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:32.907446 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:32.908081 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:32.908196 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:32.910860 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:32.910864 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:32.910864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:32.910877 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:32.912634 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:32.913047 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:32.913058 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:32.913059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:32.913155 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:32.913410 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:32.913411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:32.913411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:32.913669 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:32.913874 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:32.913965 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 16:36:32.914179 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":90,"threads_started":1}
I20260812 16:36:32.915344 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:32.923122 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:32.916833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:34541 (local address 127.0.0.1:40001)
0812 16:36:32.916877 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:32.916880 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:32.916978 (+ 98us) server_negotiation.cc:407] Connection header received
0812 16:36:32.917047 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:32.917047 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:32.917057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:32.917113 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:32.917631 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:32.918135 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:32.918864 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:32.919105 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:32.919180 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:32.920448 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:32.920450 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:32.920451 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:32.920458 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:32.921869 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:32.922260 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:32.922261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:32.922261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:32.922321 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:32.922560 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:32.922561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:32.922562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:32.922678 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:32.922788 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:32.922936 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 16:36:32.923003 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:36:32.943701 12269 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:32.944433 12269 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:32.952553 12269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 169d7781f62f4bd58ff5ae6f556ba9f4: Generated new TSK 0
I20260812 16:36:32.952771 12269 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:32.960911 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 12173
W20260812 16:36:32.968206 12155 connection.cc:535] client connection to 127.0.0.1:40001 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40001): Connection reset by peer (error 104)
2026-08-12T16:36:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:33 dist-test-slave-mvvj krb5kdc[12286](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:33 dist-test-slave-mvvj krb5kdc[12286](info): set up 2 sockets
Aug 12 16:36:33 dist-test-slave-mvvj krb5kdc[12286](info): commencing operation
krb5kdc: starting...
W20260812 16:36:34.077301 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:34 dist-test-slave-mvvj krb5kdc[12286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:36:34.211732 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43561
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:34.296975 12303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:34.297197 12303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:34.297225 12303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:34.298436 12303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:34.298486 12303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:34.298501 12303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:34.298513 12303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:34.298565 12303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:34.300119 12303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43561
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:34.300415 12303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:34.300666 12303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:34.305701 12311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:34.305724 12309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:34.305764 12303 server_base.cc:1034] running on GCE node
W20260812 16:36:34.305755 12308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:34.306285 12303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:34.306634 12303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:34.307809 12303 hybrid_clock.cc:648] HybridClock initialized: now 1786552594307777 us; error 48 us; skew 500 ppm
Aug 12 16:36:34 dist-test-slave-mvvj krb5kdc[12286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552594, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:34.310643 12303 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:36:34.311188 12303 webserver.cc:460] Webserver started at http://127.0.0.1:39511/ using document root <none> and password file <none>
I20260812 16:36:34.311455 12303 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:34.311501 12303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:34.311671 12303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:34.312897 12303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "ea670f645c2b4cc58d1642066e890da4"
format_stamp: "Formatted at 2026-08-12 16:36:34 on dist-test-slave-mvvj"
server_key: "e96f8fb6c42c6ae28571bbca0d30d474"
server_key_iv: "cf91acff75220e7621148cd872f22cce"
server_key_version: "encryptionkey@0"
I20260812 16:36:34.313284 12303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "ea670f645c2b4cc58d1642066e890da4"
format_stamp: "Formatted at 2026-08-12 16:36:34 on dist-test-slave-mvvj"
server_key: "e96f8fb6c42c6ae28571bbca0d30d474"
server_key_iv: "cf91acff75220e7621148cd872f22cce"
server_key_version: "encryptionkey@0"
I20260812 16:36:34.314678 12303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 16:36:34.315465 12319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:34.315696 12303 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:34.315796 12303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "ea670f645c2b4cc58d1642066e890da4"
format_stamp: "Formatted at 2026-08-12 16:36:34 on dist-test-slave-mvvj"
server_key: "e96f8fb6c42c6ae28571bbca0d30d474"
server_key_iv: "cf91acff75220e7621148cd872f22cce"
server_key_version: "encryptionkey@0"
I20260812 16:36:34.315932 12303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:34.333309 12303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:34.335773 12303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:34.336055 12303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:34.340317 12303 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43561
I20260812 16:36:34.340353 12381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43561 every 8 connection(s)
I20260812 16:36:34.340750 12303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:34.341665 12384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:34.344043 12384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4: Bootstrap starting.
I20260812 16:36:34.344604 12384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:34.344841 12384 log.cc:824] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:34.345562 12384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4: No bootstrap required, opened a new log
I20260812 16:36:34.347131 12384 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea670f645c2b4cc58d1642066e890da4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43561 } }
I20260812 16:36:34.347278 12384 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:34.347301 12384 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea670f645c2b4cc58d1642066e890da4, State: Initialized, Role: FOLLOWER
I20260812 16:36:34.347409 12384 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea670f645c2b4cc58d1642066e890da4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43561 } }
I20260812 16:36:34.347497 12384 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:34.347532 12384 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:34.347584 12384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:34.347983 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 12303
I20260812 16:36:34.348104 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:34.348320 20409 external_mini_cluster.cc:1428] Setting key c345a59cee0640c8af5b91e0271afe5e
I20260812 16:36:34.348371 12384 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea670f645c2b4cc58d1642066e890da4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43561 } }
I20260812 16:36:34.348488 12384 leader_election.cc:302] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea670f645c2b4cc58d1642066e890da4; no voters:
I20260812 16:36:34.348676 12384 leader_election.cc:288] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:34.348732 12387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:34.348955 12387 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4 [term 1 LEADER]: Becoming Leader. State: Replica: ea670f645c2b4cc58d1642066e890da4, State: Running, Role: LEADER
I20260812 16:36:34.349115 12387 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea670f645c2b4cc58d1642066e890da4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43561 } }
I20260812 16:36:34.349205 12384 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:34.349512 12387 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea670f645c2b4cc58d1642066e890da4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea670f645c2b4cc58d1642066e890da4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43561 } } }
I20260812 16:36:34.349675 12387 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:34.349696 12389 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea670f645c2b4cc58d1642066e890da4. Latest consensus state: current_term: 1 leader_uuid: "ea670f645c2b4cc58d1642066e890da4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea670f645c2b4cc58d1642066e890da4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43561 } } }
I20260812 16:36:34.349814 12389 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:34.350183 12400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:34.350735 12400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:34.352628 12400 catalog_manager.cc:1347] Generated new cluster ID: c1907dd3619248c48482e99fe0305346
I20260812 16:36:34.352694 12400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:34.355528 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:34.348862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:46379 (local address 127.0.0.1:43561)
0812 16:36:34.350810 (+ 1948us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:34.350822 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:34.350862 (+ 40us) server_negotiation.cc:407] Connection header received
0812 16:36:34.350996 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:34.351004 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:34.351080 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:34.351243 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:34.351457 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:34.352778 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:34.353607 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:34.354021 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:34.354150 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:34.354884 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:34.354888 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:34.354888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:34.355068 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:34.355099 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:34.355110 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:34.355205 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:34.355287 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 16:36:34.355316 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1893,"thread_start_us":82,"threads_started":1}
I20260812 16:36:34.356518 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:34.363021 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:34.358068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:38517 (local address 127.0.0.1:43561)
0812 16:36:34.358106 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:34.358108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:34.360022 (+ 1914us) server_negotiation.cc:407] Connection header received
0812 16:36:34.360092 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:34.360092 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:34.360104 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:34.360172 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:34.360320 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:34.360793 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:34.361728 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:34.361960 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:34.362050 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:34.362726 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:34.362728 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:34.362728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:34.362733 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:34.362749 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:34.362753 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:34.362826 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:34.362892 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 16:36:34.362903 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":8,"server-negotiator.queue_time_us":18}
I20260812 16:36:34.371997 12400 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:34.372552 12400 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:34.381923 12400 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea670f645c2b4cc58d1642066e890da4: Generated new TSK 0
I20260812 16:36:34.382174 12400 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:34.386622 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 12303
W20260812 16:36:34.393824 12284 connection.cc:535] client connection to 127.0.0.1:43561 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43561): Connection reset by peer (error 104)
2026-08-12T16:36:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:34 dist-test-slave-mvvj krb5kdc[12418](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:34 dist-test-slave-mvvj krb5kdc[12418](info): set up 2 sockets
Aug 12 16:36:34 dist-test-slave-mvvj krb5kdc[12418](info): commencing operation
krb5kdc: starting...
W20260812 16:36:35.512359 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:35 dist-test-slave-mvvj krb5kdc[12418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:36:35.642648 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40265
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:35.725740 12434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:35.726001 12434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:35.726049 12434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:35.727222 12434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:35.727291 12434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:35.727314 12434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:35.727334 12434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:35.727366 12434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:35.728888 12434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40265
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:35.729187 12434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:35.729418 12434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:35.734377 12441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:35.734404 12443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:35.734663 12434 server_base.cc:1034] running on GCE node
W20260812 16:36:35.734417 12440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:35.736105 12434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:35.736383 12434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:35.737562 12434 hybrid_clock.cc:648] HybridClock initialized: now 1786552595737546 us; error 29 us; skew 500 ppm
Aug 12 16:36:35 dist-test-slave-mvvj krb5kdc[12418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552595, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:35.740131 12434 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:36:35.740532 12434 webserver.cc:460] Webserver started at http://127.0.0.1:35135/ using document root <none> and password file <none>
I20260812 16:36:35.740727 12434 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:35.740767 12434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:35.740856 12434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:35.741876 12434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "cefd27e42e574879af8d1a0ce36b6fb0"
format_stamp: "Formatted at 2026-08-12 16:36:35 on dist-test-slave-mvvj"
server_key: "0f15e8bf2fe6a55326a0fb26e24ab4a3"
server_key_iv: "cf1aa12ce40c3d54e85161a20946e10d"
server_key_version: "encryptionkey@0"
I20260812 16:36:35.742183 12434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "cefd27e42e574879af8d1a0ce36b6fb0"
format_stamp: "Formatted at 2026-08-12 16:36:35 on dist-test-slave-mvvj"
server_key: "0f15e8bf2fe6a55326a0fb26e24ab4a3"
server_key_iv: "cf1aa12ce40c3d54e85161a20946e10d"
server_key_version: "encryptionkey@0"
I20260812 16:36:35.743498 12434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:36:35.744246 12450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:35.744451 12434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:35.744537 12434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "cefd27e42e574879af8d1a0ce36b6fb0"
format_stamp: "Formatted at 2026-08-12 16:36:35 on dist-test-slave-mvvj"
server_key: "0f15e8bf2fe6a55326a0fb26e24ab4a3"
server_key_iv: "cf1aa12ce40c3d54e85161a20946e10d"
server_key_version: "encryptionkey@0"
I20260812 16:36:35.744596 12434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:35.747900 12434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:35.749811 12434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:35.749967 12434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:35.753901 12434 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40265
I20260812 16:36:35.753947 12506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40265 every 8 connection(s)
I20260812 16:36:35.754282 12434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:35.755270 12507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:35.757465 12507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0: Bootstrap starting.
I20260812 16:36:35.757972 12507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:35.758258 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 12434
I20260812 16:36:35.758328 12507 log.cc:824] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:35.758411 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:35.758630 20409 external_mini_cluster.cc:1428] Setting key 253fc29505cc8f790c8ad10cc8609e89
I20260812 16:36:35.759195 12507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0: No bootstrap required, opened a new log
I20260812 16:36:35.761410 12507 raft_consensus.cc:348] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefd27e42e574879af8d1a0ce36b6fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40265 } }
I20260812 16:36:35.761637 12507 raft_consensus.cc:374] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:35.761690 12507 raft_consensus.cc:729] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cefd27e42e574879af8d1a0ce36b6fb0, State: Initialized, Role: FOLLOWER
I20260812 16:36:35.761837 12507 consensus_queue.cc:260] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefd27e42e574879af8d1a0ce36b6fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40265 } }
I20260812 16:36:35.761940 12507 raft_consensus.cc:388] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:35.761976 12507 raft_consensus.cc:482] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:35.762001 12511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:35.759163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:38069 (local address 127.0.0.1:40265)
0812 16:36:35.759358 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:35.759367 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:35.759531 (+ 164us) server_negotiation.cc:407] Connection header received
0812 16:36:35.760523 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:35.760532 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:35.760567 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:35.760710 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:35.761356 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:35.761359 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:35.761360 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:35.761364 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:35.761408 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:35.761423 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:35.761499 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:35.761553 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 16:36:35.761595 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":73,"threads_started":1}
I20260812 16:36:35.762030 12507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:35.762763 12507 raft_consensus.cc:504] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefd27e42e574879af8d1a0ce36b6fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40265 } }
I20260812 16:36:35.762889 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:35.762872 12507 leader_election.cc:302] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cefd27e42e574879af8d1a0ce36b6fb0; no voters:
I20260812 16:36:35.763053 12507 leader_election.cc:288] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:35.763085 12512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:35.763195 12512 raft_consensus.cc:686] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0 [term 1 LEADER]: Becoming Leader. State: Replica: cefd27e42e574879af8d1a0ce36b6fb0, State: Running, Role: LEADER
I20260812 16:36:35.763309 12507 sys_catalog.cc:564] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:35.763334 12512 consensus_queue.cc:237] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefd27e42e574879af8d1a0ce36b6fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40265 } }
I20260812 16:36:35.763686 12512 sys_catalog.cc:455] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cefd27e42e574879af8d1a0ce36b6fb0. Latest consensus state: current_term: 1 leader_uuid: "cefd27e42e574879af8d1a0ce36b6fb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefd27e42e574879af8d1a0ce36b6fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40265 } } }
I20260812 16:36:35.763702 12513 sys_catalog.cc:455] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cefd27e42e574879af8d1a0ce36b6fb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefd27e42e574879af8d1a0ce36b6fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40265 } } }
I20260812 16:36:35.763881 12512 sys_catalog.cc:458] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:35.763938 12513 sys_catalog.cc:458] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:35.764262 12525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:35.764840 12525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:35.765802 12511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:35.764479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:58523 (local address 127.0.0.1:40265)
0812 16:36:35.764516 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:35.764518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:35.764689 (+ 171us) server_negotiation.cc:407] Connection header received
0812 16:36:35.764792 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:35.764793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:35.764798 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:35.764828 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:35.765556 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:35.765558 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:35.765558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:35.765563 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:35.765577 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:35.765581 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:35.765612 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:35.765724 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 16:36:35.765739 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:36:35.767103 12525 catalog_manager.cc:1347] Generated new cluster ID: 28c60449bec94c5384a2a6f4d2181869
I20260812 16:36:35.767174 12525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:35.784046 12525 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:35.784235 12525 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:35.789760 12525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cefd27e42e574879af8d1a0ce36b6fb0: Generated new TSK 0
I20260812 16:36:35.789951 12525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:35.796155 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 12434
2026-08-12T16:36:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1414 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:35 dist-test-slave-mvvj krb5kdc[12543](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:35 dist-test-slave-mvvj krb5kdc[12543](info): set up 2 sockets
Aug 12 16:36:35 dist-test-slave-mvvj krb5kdc[12543](info): commencing operation
krb5kdc: starting...
W20260812 16:36:36.901621 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:36 dist-test-slave-mvvj krb5kdc[12543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:36:37.034889 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:38761
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:45645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:38761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:37.120954 12557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:37.121172 12557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:37.121196 12557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:37.122377 12557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:37.122430 12557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:37.122445 12557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:37.122458 12557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:37.122470 12557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:37.123942 12557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:45645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:38761
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:38761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:37.124151 12557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:37.124329 12557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:37.129421 12563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:37.129400 12566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:37.129391 12562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:37.129748 12557 server_base.cc:1034] running on GCE node
I20260812 16:36:37.130016 12557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:37.130322 12557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:37.131495 12557 hybrid_clock.cc:648] HybridClock initialized: now 1786552597131505 us; error 73 us; skew 500 ppm
Aug 12 16:36:37 dist-test-slave-mvvj krb5kdc[12543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552597, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:37.137861 12557 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:36:37.138502 12557 webserver.cc:460] Webserver started at http://127.19.238.126:33831/ using document root <none> and password file <none>
I20260812 16:36:37.138756 12557 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:37.138827 12557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:37.138988 12557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:37.140195 12557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "7839738e217f46b1bea84549aa9723bc"
format_stamp: "Formatted at 2026-08-12 16:36:37 on dist-test-slave-mvvj"
server_key: "1049487af9389263e3a82f6f49fcca09"
server_key_iv: "6cda86e32271a45c505c3a3878a966e1"
server_key_version: "encryptionkey@0"
I20260812 16:36:37.140573 12557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "7839738e217f46b1bea84549aa9723bc"
format_stamp: "Formatted at 2026-08-12 16:36:37 on dist-test-slave-mvvj"
server_key: "1049487af9389263e3a82f6f49fcca09"
server_key_iv: "6cda86e32271a45c505c3a3878a966e1"
server_key_version: "encryptionkey@0"
I20260812 16:36:37.142015 12557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 16:36:37.142961 12574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:37.143247 12557 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:37.143415 12557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "7839738e217f46b1bea84549aa9723bc"
format_stamp: "Formatted at 2026-08-12 16:36:37 on dist-test-slave-mvvj"
server_key: "1049487af9389263e3a82f6f49fcca09"
server_key_iv: "6cda86e32271a45c505c3a3878a966e1"
server_key_version: "encryptionkey@0"
I20260812 16:36:37.143514 12557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:37.174481 12557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:37.176721 12557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:37.176918 12557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:37.181725 12557 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:38761
I20260812 16:36:37.181785 12638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:38761 every 8 connection(s)
I20260812 16:36:37.182204 12557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:37.183120 12639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:37.185457 12639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc: Bootstrap starting.
I20260812 16:36:37.186064 12639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:37.186357 12639 log.cc:824] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:37.187146 12639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc: No bootstrap required, opened a new log
I20260812 16:36:37.188833 12639 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7839738e217f46b1bea84549aa9723bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38761 } }
I20260812 16:36:37.188983 12639 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:37.189006 12639 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7839738e217f46b1bea84549aa9723bc, State: Initialized, Role: FOLLOWER
I20260812 16:36:37.189173 12639 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7839738e217f46b1bea84549aa9723bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38761 } }
I20260812 16:36:37.189237 12639 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:37.189288 12639 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:37.189339 12639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:37.190253 12639 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7839738e217f46b1bea84549aa9723bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38761 } }
I20260812 16:36:37.190344 12639 leader_election.cc:302] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7839738e217f46b1bea84549aa9723bc; no voters:
I20260812 16:36:37.190538 12639 leader_election.cc:288] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:37.190650 12642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:37.190840 12639 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:37.190871 12642 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc [term 1 LEADER]: Becoming Leader. State: Replica: 7839738e217f46b1bea84549aa9723bc, State: Running, Role: LEADER
I20260812 16:36:37.191044 12642 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7839738e217f46b1bea84549aa9723bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38761 } }
I20260812 16:36:37.191175 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 12557
I20260812 16:36:37.191288 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:37.191536 20409 external_mini_cluster.cc:1428] Setting key 3a636250d312b849c982054563d6e023
I20260812 16:36:37.191814 12644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7839738e217f46b1bea84549aa9723bc. Latest consensus state: current_term: 1 leader_uuid: "7839738e217f46b1bea84549aa9723bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7839738e217f46b1bea84549aa9723bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38761 } } }
I20260812 16:36:37.191941 12644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:37.192272 12651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:37.192611 12643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7839738e217f46b1bea84549aa9723bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7839738e217f46b1bea84549aa9723bc" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38761 } } }
I20260812 16:36:37.192713 12643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:37.193006 12651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:37.195590 12651 catalog_manager.cc:1347] Generated new cluster ID: 1816dbd94b1e4f5f8592073e76fc62b6
I20260812 16:36:37.195652 12651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 16:36:37 dist-test-slave-mvvj krb5kdc[12543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:36:37.208745 12659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:37.192280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57343 (local address 127.19.238.126:38761)
0812 16:36:37.193653 (+ 1373us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:37.193663 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:37.193698 (+ 35us) server_negotiation.cc:407] Connection header received
0812 16:36:37.193821 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:37.193828 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:37.193913 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:37.194084 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:37.194934 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:37.196200 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:37.196996 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:37.197523 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:37.198019 (+ 496us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:37.204290 (+ 6271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:37.204296 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:37.204296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:37.204312 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:37.206762 (+ 2450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:37.207372 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:37.207375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:37.207375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:37.207464 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:37.207723 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:37.207724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:37.207725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:37.208035 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:37.208209 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:37.208348 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 16:36:37.208458 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1321,"thread_start_us":61,"threads_started":1}
I20260812 16:36:37.209167 12651 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:37.209985 12651 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:37.210798 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:37.220494 12659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:37.212365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34265 (local address 127.19.238.126:38761)
0812 16:36:37.212405 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:37.212408 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:37.212531 (+ 123us) server_negotiation.cc:407] Connection header received
0812 16:36:37.212602 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:37.212603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:37.212615 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:37.212686 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:37.213365 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:37.214213 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:37.215302 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:37.215655 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:37.215724 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:37.217330 (+ 1606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:37.217332 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:37.217332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:37.217340 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:37.219118 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:37.219571 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:37.219572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:37.219572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:37.219637 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:37.219909 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:37.219910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:37.219911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:37.220046 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:37.220156 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:37.220314 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 16:36:37.220383 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:36:37.221853 12651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7839738e217f46b1bea84549aa9723bc: Generated new TSK 0
I20260812 16:36:37.222067 12651 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:37.225557 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 12557
W20260812 16:36:37.232172 12541 connection.cc:535] client connection to 127.19.238.126:38761 recv error: Network error: failed to read from TLS socket (remote: 127.19.238.126:38761): Connection reset by peer (error 104)
2026-08-12T16:36:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:37 dist-test-slave-mvvj krb5kdc[12671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:37 dist-test-slave-mvvj krb5kdc[12671](info): set up 2 sockets
Aug 12 16:36:37 dist-test-slave-mvvj krb5kdc[12671](info): commencing operation
krb5kdc: starting...
W20260812 16:36:38.336779 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:38 dist-test-slave-mvvj krb5kdc[12671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:36:38.483913 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:34667
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:35439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:34667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:38.570798 12686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:38.571076 12686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:38.571121 12686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:38.572408 12686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:38.572456 12686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:38.572471 12686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:38.572484 12686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:38.572504 12686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:38.573992 12686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:35439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:34667
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:34667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:38.574203 12686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:38.574383 12686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:38.579500 12693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:38.579530 12695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:38.579562 12686 server_base.cc:1034] running on GCE node
W20260812 16:36:38.579500 12697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:38.580050 12686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:38.580359 12686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:38.581530 12686 hybrid_clock.cc:648] HybridClock initialized: now 1786552598581520 us; error 48 us; skew 500 ppm
Aug 12 16:36:38 dist-test-slave-mvvj krb5kdc[12671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552598, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:38.584245 12686 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:36:38.584726 12686 webserver.cc:460] Webserver started at http://127.19.238.126:37321/ using document root <none> and password file <none>
I20260812 16:36:38.584964 12686 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:38.585035 12686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:38.585189 12686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:38.586377 12686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "bd2ae9d95b6546b0b07f7c2e0c2d1080"
format_stamp: "Formatted at 2026-08-12 16:36:38 on dist-test-slave-mvvj"
server_key: "5c159d76007cfb8a764e49b78f323a5c"
server_key_iv: "2571bf8887eac016082638cab1205ffd"
server_key_version: "encryptionkey@0"
I20260812 16:36:38.586782 12686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "bd2ae9d95b6546b0b07f7c2e0c2d1080"
format_stamp: "Formatted at 2026-08-12 16:36:38 on dist-test-slave-mvvj"
server_key: "5c159d76007cfb8a764e49b78f323a5c"
server_key_iv: "2571bf8887eac016082638cab1205ffd"
server_key_version: "encryptionkey@0"
I20260812 16:36:38.588250 12686 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:36:38.589135 12705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:38.589354 12686 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:36:38.589471 12686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "bd2ae9d95b6546b0b07f7c2e0c2d1080"
format_stamp: "Formatted at 2026-08-12 16:36:38 on dist-test-slave-mvvj"
server_key: "5c159d76007cfb8a764e49b78f323a5c"
server_key_iv: "2571bf8887eac016082638cab1205ffd"
server_key_version: "encryptionkey@0"
I20260812 16:36:38.589577 12686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:38.608788 12686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:38.611469 12686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:38.611651 12686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:38.616442 12686 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:34667
I20260812 16:36:38.616590 12760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:34667 every 8 connection(s)
I20260812 16:36:38.616923 12686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:38.617965 12761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:38.620074 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 12686
I20260812 16:36:38.620206 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:38.620424 20409 external_mini_cluster.cc:1428] Setting key 763fb75c2a56d1a05c64639da5181076
I20260812 16:36:38.620916 12761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080: Bootstrap starting.
I20260812 16:36:38.621639 12761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:38.621974 12761 log.cc:824] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:38.622828 12761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080: No bootstrap required, opened a new log
I20260812 16:36:38.625505 12761 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2ae9d95b6546b0b07f7c2e0c2d1080" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 34667 } }
I20260812 16:36:38.625701 12761 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:38.625743 12761 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd2ae9d95b6546b0b07f7c2e0c2d1080, State: Initialized, Role: FOLLOWER
I20260812 16:36:38.625885 12761 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2ae9d95b6546b0b07f7c2e0c2d1080" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 34667 } }
I20260812 16:36:38.625981 12761 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:38.626021 12761 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:38.626075 12761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:38.626650 12765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:38.621066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56091 (local address 127.19.238.126:34667)
0812 16:36:38.621266 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:38.621274 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:38.621302 (+ 28us) server_negotiation.cc:407] Connection header received
0812 16:36:38.621850 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:38.621858 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:38.621911 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:38.622037 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:38.622322 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:38.623402 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:38.624568 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:38.624995 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:38.625092 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:38.625916 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:38.625920 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:38.625920 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:38.626142 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:38.626185 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:38.626200 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:38.626296 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:38.626399 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 16:36:38.626429 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":92,"threads_started":1}
I20260812 16:36:38.626900 12761 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2ae9d95b6546b0b07f7c2e0c2d1080" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 34667 } }
I20260812 16:36:38.627028 12761 leader_election.cc:302] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd2ae9d95b6546b0b07f7c2e0c2d1080; no voters:
I20260812 16:36:38.627203 12761 leader_election.cc:288] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:38.627259 12769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:38.627458 12769 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080 [term 1 LEADER]: Becoming Leader. State: Replica: bd2ae9d95b6546b0b07f7c2e0c2d1080, State: Running, Role: LEADER
I20260812 16:36:38.627487 12761 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:38.627575 12769 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2ae9d95b6546b0b07f7c2e0c2d1080" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 34667 } }
I20260812 16:36:38.627753 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:38.628665 12772 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd2ae9d95b6546b0b07f7c2e0c2d1080. Latest consensus state: current_term: 1 leader_uuid: "bd2ae9d95b6546b0b07f7c2e0c2d1080" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2ae9d95b6546b0b07f7c2e0c2d1080" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 34667 } } }
I20260812 16:36:38.628757 12772 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:38.629451 12770 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd2ae9d95b6546b0b07f7c2e0c2d1080" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2ae9d95b6546b0b07f7c2e0c2d1080" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 34667 } } }
W20260812 16:36:38.629501 12787 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260812 16:36:38.629544 12770 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080 [sys.catalog]: This master's current role is: LEADER
W20260812 16:36:38.629554 12787 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:36:38.629840 12786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:38.630079 12786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:38.631707 12786 catalog_manager.cc:1347] Generated new cluster ID: ee9efc5aa735497492389743afbdb183
I20260812 16:36:38.631762 12786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:38.633582 12765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:38.629892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50187 (local address 127.19.238.126:34667)
0812 16:36:38.629932 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:38.629934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:38.630113 (+ 179us) server_negotiation.cc:407] Connection header received
0812 16:36:38.630219 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:38.630220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:38.630231 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:38.630314 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:38.630463 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:38.631185 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:38.632160 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:38.632388 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:38.632472 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:38.633261 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:38.633262 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:38.633262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:38.633268 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:38.633284 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:38.633289 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:38.633348 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:38.633459 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 16:36:38.633470 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 16:36:38.649880 12786 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:38.650532 12786 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:38.654934 12786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd2ae9d95b6546b0b07f7c2e0c2d1080: Generated new TSK 0
I20260812 16:36:38.655145 12786 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:38.665019 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 12686
W20260812 16:36:38.673112 12669 connection.cc:535] client connection to 127.19.238.126:34667 recv error: Network error: failed to read from TLS socket (remote: 127.19.238.126:34667): Connection reset by peer (error 104)
2026-08-12T16:36:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:38 dist-test-slave-mvvj krb5kdc[12799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:38 dist-test-slave-mvvj krb5kdc[12799](info): set up 2 sockets
Aug 12 16:36:38 dist-test-slave-mvvj krb5kdc[12799](info): commencing operation
krb5kdc: starting...
W20260812 16:36:39.786790 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:39 dist-test-slave-mvvj krb5kdc[12799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:36:39.915305 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:37211
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:46543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:37211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:39.998145 12813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:39.998405 12813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:39.998445 12813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:39.999598 12813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:39.999671 12813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:39.999699 12813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:39.999733 12813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:39.999758 12813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:40.001322 12813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:46543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:37211
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:37211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:40.001616 12813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:40.001854 12813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:40.006880 12820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:40.006875 12822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:40.006899 12824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:40.007180 12813 server_base.cc:1034] running on GCE node
I20260812 16:36:40.007436 12813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:40.007756 12813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:40.008924 12813 hybrid_clock.cc:648] HybridClock initialized: now 1786552600008925 us; error 61 us; skew 500 ppm
Aug 12 16:36:40 dist-test-slave-mvvj krb5kdc[12799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552600, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:40.011539 12813 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:36:40.012087 12813 webserver.cc:460] Webserver started at http://127.19.238.126:40313/ using document root <none> and password file <none>
I20260812 16:36:40.012346 12813 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:40.012416 12813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:40.012575 12813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:40.013679 12813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "9ba636470008474388fe52518ec09021"
format_stamp: "Formatted at 2026-08-12 16:36:40 on dist-test-slave-mvvj"
server_key: "fa1cace78b6c089cf37d017295981a30"
server_key_iv: "d9509589e0f98e4adf31afca03a9c6a7"
server_key_version: "encryptionkey@0"
I20260812 16:36:40.014073 12813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "9ba636470008474388fe52518ec09021"
format_stamp: "Formatted at 2026-08-12 16:36:40 on dist-test-slave-mvvj"
server_key: "fa1cace78b6c089cf37d017295981a30"
server_key_iv: "d9509589e0f98e4adf31afca03a9c6a7"
server_key_version: "encryptionkey@0"
I20260812 16:36:40.015447 12813 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:36:40.016260 12831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:40.016494 12813 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:36:40.016590 12813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "9ba636470008474388fe52518ec09021"
format_stamp: "Formatted at 2026-08-12 16:36:40 on dist-test-slave-mvvj"
server_key: "fa1cace78b6c089cf37d017295981a30"
server_key_iv: "d9509589e0f98e4adf31afca03a9c6a7"
server_key_version: "encryptionkey@0"
I20260812 16:36:40.016677 12813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:40.020053 12813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:40.022063 12813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:40.022224 12813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:40.026420 12813 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:37211
I20260812 16:36:40.026458 12890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:37211 every 8 connection(s)
I20260812 16:36:40.026816 12813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:40.027917 12891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:40.030215 12891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021: Bootstrap starting.
I20260812 16:36:40.030838 12891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:40.030889 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 12813
I20260812 16:36:40.031023 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:40.031193 12891 log.cc:824] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:40.031226 20409 external_mini_cluster.cc:1428] Setting key d03686cda14622b6d9572b58bfb2301a
I20260812 16:36:40.032166 12891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021: No bootstrap required, opened a new log
I20260812 16:36:40.033975 12891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba636470008474388fe52518ec09021" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37211 } }
I20260812 16:36:40.034071 12897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:40.031741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42785 (local address 127.19.238.126:37211)
0812 16:36:40.031908 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:40.031919 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:40.031957 (+ 38us) server_negotiation.cc:407] Connection header received
0812 16:36:40.032479 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:40.032486 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:40.032513 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:40.032668 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:40.033398 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:40.033401 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:40.033401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:40.033405 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:40.033443 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:40.033454 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:40.033556 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:40.033657 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 16:36:40.033702 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":62,"threads_started":1}
I20260812 16:36:40.034142 12891 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:40.034188 12891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ba636470008474388fe52518ec09021, State: Initialized, Role: FOLLOWER
I20260812 16:36:40.034308 12891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba636470008474388fe52518ec09021" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37211 } }
I20260812 16:36:40.034420 12891 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:40.034456 12891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:40.034502 12891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:40.035054 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:40.035238 12891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba636470008474388fe52518ec09021" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37211 } }
I20260812 16:36:40.035341 12891 leader_election.cc:302] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ba636470008474388fe52518ec09021; no voters:
I20260812 16:36:40.035490 12891 leader_election.cc:288] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:40.035527 12898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:40.035733 12898 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021 [term 1 LEADER]: Becoming Leader. State: Replica: 9ba636470008474388fe52518ec09021, State: Running, Role: LEADER
I20260812 16:36:40.035748 12891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:40.035897 12898 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba636470008474388fe52518ec09021" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37211 } }
I20260812 16:36:40.036993 12900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ba636470008474388fe52518ec09021" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba636470008474388fe52518ec09021" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37211 } } }
I20260812 16:36:40.037108 12900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:40.037335 12919 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:40.037469 12899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ba636470008474388fe52518ec09021. Latest consensus state: current_term: 1 leader_uuid: "9ba636470008474388fe52518ec09021" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba636470008474388fe52518ec09021" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 37211 } } }
I20260812 16:36:40.038033 12899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:40.038282 12897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:40.036865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38611 (local address 127.19.238.126:37211)
0812 16:36:40.037279 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:40.037281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:40.037290 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:36:40.037309 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:40.037310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:40.037314 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:40.037405 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:40.037981 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:40.037982 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:40.037983 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:40.037987 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:40.038004 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:40.038008 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:40.038063 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:40.038201 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 16:36:40.038216 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394}
I20260812 16:36:40.038486 12919 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:40.040218 12919 catalog_manager.cc:1347] Generated new cluster ID: 4f4a0c55fbc54beead608e5a45e3e8dc
I20260812 16:36:40.040271 12919 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:40.050920 12919 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:40.051122 12919 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:40.055131 12919 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ba636470008474388fe52518ec09021: Generated new TSK 0
I20260812 16:36:40.055348 12919 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:40.064448 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 12813
2026-08-12T16:36:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1399 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:40 dist-test-slave-mvvj krb5kdc[12927](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:40 dist-test-slave-mvvj krb5kdc[12927](info): set up 2 sockets
Aug 12 16:36:40 dist-test-slave-mvvj krb5kdc[12927](info): commencing operation
krb5kdc: starting...
W20260812 16:36:41.205335 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:41 dist-test-slave-mvvj krb5kdc[12927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:36:41.338876 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:35135
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:37797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:35135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:41.422706 12944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:41.423032 12944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:41.423069 12944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:41.424602 12944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:41.424651 12944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:41.424666 12944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:41.424679 12944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:41.424691 12944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:41.426218 12944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:37797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:35135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:35135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:41.426486 12944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:41.426723 12944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:41.431797 12953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:41.431886 12950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:41.431874 12951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:41.431869 12944 server_base.cc:1034] running on GCE node
I20260812 16:36:41.432308 12944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:41.432651 12944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:41.433856 12944 hybrid_clock.cc:648] HybridClock initialized: now 1786552601433794 us; error 82 us; skew 500 ppm
Aug 12 16:36:41 dist-test-slave-mvvj krb5kdc[12927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552601, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:41.436709 12944 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:36:41.437250 12944 webserver.cc:460] Webserver started at http://127.19.238.126:43539/ using document root <none> and password file <none>
I20260812 16:36:41.437467 12944 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:41.437510 12944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:41.437608 12944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:41.438704 12944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "73eb3e770f2a4569a399ae6b28d4a533"
format_stamp: "Formatted at 2026-08-12 16:36:41 on dist-test-slave-mvvj"
server_key: "4f49d653829a28eb2ae3c6af815bdb6b"
server_key_iv: "b71b36ffddac1d2f20edf44e7addd2c1"
server_key_version: "encryptionkey@0"
I20260812 16:36:41.439070 12944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "73eb3e770f2a4569a399ae6b28d4a533"
format_stamp: "Formatted at 2026-08-12 16:36:41 on dist-test-slave-mvvj"
server_key: "4f49d653829a28eb2ae3c6af815bdb6b"
server_key_iv: "b71b36ffddac1d2f20edf44e7addd2c1"
server_key_version: "encryptionkey@0"
I20260812 16:36:41.440716 12944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 16:36:41.441605 12960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:41.441845 12944 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:41.441947 12944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "73eb3e770f2a4569a399ae6b28d4a533"
format_stamp: "Formatted at 2026-08-12 16:36:41 on dist-test-slave-mvvj"
server_key: "4f49d653829a28eb2ae3c6af815bdb6b"
server_key_iv: "b71b36ffddac1d2f20edf44e7addd2c1"
server_key_version: "encryptionkey@0"
I20260812 16:36:41.442022 12944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:41.462322 12944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:41.466655 12944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:41.466917 12944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:41.471628 12944 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:35135
I20260812 16:36:41.471705 13021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:35135 every 8 connection(s)
I20260812 16:36:41.472229 12944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:41.473438 13022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:41.475031 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 12944
I20260812 16:36:41.475157 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:41.475360 20409 external_mini_cluster.cc:1428] Setting key 6563fc79a8b002c100c9ec85ab71f141
I20260812 16:36:41.477123 13022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533: Bootstrap starting.
I20260812 16:36:41.477994 13022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:41.478343 13022 log.cc:824] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:41.479128 13022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533: No bootstrap required, opened a new log
I20260812 16:36:41.481878 13022 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73eb3e770f2a4569a399ae6b28d4a533" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 35135 } }
I20260812 16:36:41.482089 13022 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:41.482125 13022 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73eb3e770f2a4569a399ae6b28d4a533, State: Initialized, Role: FOLLOWER
I20260812 16:36:41.482256 13022 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73eb3e770f2a4569a399ae6b28d4a533" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 35135 } }
I20260812 16:36:41.482354 13022 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:41.482402 13022 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:41.482455 13022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 16:36:41 dist-test-slave-mvvj krb5kdc[12927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:36:41.483237 13022 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73eb3e770f2a4569a399ae6b28d4a533" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 35135 } }
I20260812 16:36:41.483368 13022 leader_election.cc:302] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73eb3e770f2a4569a399ae6b28d4a533; no voters:
I20260812 16:36:41.483564 13022 leader_election.cc:288] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:41.483631 13027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:41.483875 13027 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533 [term 1 LEADER]: Becoming Leader. State: Replica: 73eb3e770f2a4569a399ae6b28d4a533, State: Running, Role: LEADER
I20260812 16:36:41.483952 13022 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:41.484031 13027 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73eb3e770f2a4569a399ae6b28d4a533" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 35135 } }
I20260812 16:36:41.484330 13027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73eb3e770f2a4569a399ae6b28d4a533. Latest consensus state: current_term: 1 leader_uuid: "73eb3e770f2a4569a399ae6b28d4a533" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73eb3e770f2a4569a399ae6b28d4a533" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 35135 } } }
I20260812 16:36:41.484390 13027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:41.484342 13028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73eb3e770f2a4569a399ae6b28d4a533" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73eb3e770f2a4569a399ae6b28d4a533" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 35135 } } }
I20260812 16:36:41.484625 13032 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:41.484694 13028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:41.485469 13032 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:41.487496 13032 catalog_manager.cc:1347] Generated new cluster ID: d3777a1f92814cc591895073f04c2c29
I20260812 16:36:41.487632 13032 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:41.488850 13025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:41.476283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42703 (local address 127.19.238.126:35135)
0812 16:36:41.476474 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:41.476484 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:41.476527 (+ 43us) server_negotiation.cc:407] Connection header received
0812 16:36:41.476950 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:41.476956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:41.477002 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:41.477125 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:41.477999 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:41.479164 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:41.480036 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:41.480506 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:41.480657 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:41.483677 (+ 3020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:41.483682 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:41.483682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:41.483696 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:41.486711 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:41.487171 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:41.487175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:41.487176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:41.487299 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:41.487513 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:41.487514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:41.487514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:41.487861 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:41.488010 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:41.488480 (+ 470us) server_negotiation.cc:299] Negotiation successful
0812 16:36:41.488626 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":105,"threads_started":1}
I20260812 16:36:41.489890 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:41.500414 13025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:41.491574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37311 (local address 127.19.238.126:35135)
0812 16:36:41.491615 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:41.491617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:41.491886 (+ 269us) server_negotiation.cc:407] Connection header received
0812 16:36:41.491915 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:41.491916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:41.491927 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:41.492018 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:41.492621 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:41.493408 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:41.494375 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:41.494710 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:41.494795 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:41.496146 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:41.496148 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:41.496148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:41.496156 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:41.499055 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:41.499428 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:41.499430 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:41.499430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:41.499499 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:41.499730 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:41.499731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:41.499731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:41.499905 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:41.500000 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:41.500226 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 16:36:41.500312 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:36:41.506595 13032 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:41.507128 13032 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:41.526540 13032 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73eb3e770f2a4569a399ae6b28d4a533: Generated new TSK 0
I20260812 16:36:41.526770 13032 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:41.531757 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 12944
2026-08-12T16:36:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1467 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:41 dist-test-slave-mvvj krb5kdc[13056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:41 dist-test-slave-mvvj krb5kdc[13056](info): set up 2 sockets
Aug 12 16:36:41 dist-test-slave-mvvj krb5kdc[13056](info): commencing operation
krb5kdc: starting...
W20260812 16:36:42.647281 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:42 dist-test-slave-mvvj krb5kdc[13056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:36:42.783299 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:36187
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:40787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:36187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:42.867621 13072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:42.867933 13072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:42.867975 13072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:42.869175 13072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:42.869254 13072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:42.869279 13072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:42.869302 13072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:42.869331 13072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:42.870924 13072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:40787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:36187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:36187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:42.871225 13072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:42.871481 13072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:42.876626 13080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:42.876588 13078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:42.876591 13077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:42.876753 13072 server_base.cc:1034] running on GCE node
I20260812 16:36:42.877135 13072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:42.877446 13072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:42.878643 13072 hybrid_clock.cc:648] HybridClock initialized: now 1786552602878616 us; error 44 us; skew 500 ppm
Aug 12 16:36:42 dist-test-slave-mvvj krb5kdc[13056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552602, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:42.886754 13072 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:36:42.887584 13072 webserver.cc:460] Webserver started at http://127.19.238.126:39303/ using document root <none> and password file <none>
I20260812 16:36:42.887948 13072 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:42.888025 13072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:42.888192 13072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:42.889420 13072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "6071e8a7b04b4951bbb8688a5ab170ea"
format_stamp: "Formatted at 2026-08-12 16:36:42 on dist-test-slave-mvvj"
server_key: "a36fb207a88fbfcc9de00612d94b241c"
server_key_iv: "59512694dd46b3bfb76c4707dbdc4ddf"
server_key_version: "encryptionkey@0"
I20260812 16:36:42.889832 13072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "6071e8a7b04b4951bbb8688a5ab170ea"
format_stamp: "Formatted at 2026-08-12 16:36:42 on dist-test-slave-mvvj"
server_key: "a36fb207a88fbfcc9de00612d94b241c"
server_key_iv: "59512694dd46b3bfb76c4707dbdc4ddf"
server_key_version: "encryptionkey@0"
I20260812 16:36:42.891325 13072 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:36:42.892241 13088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:42.892419 13072 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:36:42.892539 13072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "6071e8a7b04b4951bbb8688a5ab170ea"
format_stamp: "Formatted at 2026-08-12 16:36:42 on dist-test-slave-mvvj"
server_key: "a36fb207a88fbfcc9de00612d94b241c"
server_key_iv: "59512694dd46b3bfb76c4707dbdc4ddf"
server_key_version: "encryptionkey@0"
I20260812 16:36:42.892629 13072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:42.904899 13072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:42.907524 13072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:42.907713 13072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:42.911913 13072 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:36187
I20260812 16:36:42.911959 13147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:36187 every 8 connection(s)
I20260812 16:36:42.912397 13072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:42.913307 13148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:42.915827 13148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea: Bootstrap starting.
I20260812 16:36:42.916507 13148 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:42.916826 13148 log.cc:824] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:42.917603 13148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea: No bootstrap required, opened a new log
I20260812 16:36:42.919198 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 13072
I20260812 16:36:42.919351 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:42.919241 13148 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6071e8a7b04b4951bbb8688a5ab170ea" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 36187 } }
I20260812 16:36:42.919453 13148 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:42.919500 13148 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6071e8a7b04b4951bbb8688a5ab170ea, State: Initialized, Role: FOLLOWER
I20260812 16:36:42.919585 20409 external_mini_cluster.cc:1428] Setting key 8945982d82a595e6b7ca2c38f3610e36
I20260812 16:36:42.919682 13148 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6071e8a7b04b4951bbb8688a5ab170ea" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 36187 } }
I20260812 16:36:42.919775 13148 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:42.919821 13148 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:42.919903 13148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:42.921066 13148 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6071e8a7b04b4951bbb8688a5ab170ea" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 36187 } }
I20260812 16:36:42.921203 13148 leader_election.cc:302] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6071e8a7b04b4951bbb8688a5ab170ea; no voters:
I20260812 16:36:42.921360 13148 leader_election.cc:288] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:42.921429 13153 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:42.921607 13148 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:42.921648 13153 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea [term 1 LEADER]: Becoming Leader. State: Replica: 6071e8a7b04b4951bbb8688a5ab170ea, State: Running, Role: LEADER
I20260812 16:36:42.921847 13153 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6071e8a7b04b4951bbb8688a5ab170ea" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 36187 } }
I20260812 16:36:42.922231 13154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6071e8a7b04b4951bbb8688a5ab170ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6071e8a7b04b4951bbb8688a5ab170ea" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 36187 } } }
I20260812 16:36:42.922325 13154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:42.922498 13155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6071e8a7b04b4951bbb8688a5ab170ea. Latest consensus state: current_term: 1 leader_uuid: "6071e8a7b04b4951bbb8688a5ab170ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6071e8a7b04b4951bbb8688a5ab170ea" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 36187 } } }
I20260812 16:36:42.922554 13155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:42.922894 13162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:42.923516 13162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:42.925228 13162 catalog_manager.cc:1347] Generated new cluster ID: 67066ce6bc3c4c23ac122102ff1401f9
I20260812 16:36:42.925294 13162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:42.926041 13152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:42.920345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53809 (local address 127.19.238.126:36187)
0812 16:36:42.920539 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:42.920552 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:42.920594 (+ 42us) server_negotiation.cc:407] Connection header received
0812 16:36:42.920745 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:42.920753 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:42.920812 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:42.920960 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:42.921187 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:42.922608 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:42.923729 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:42.924351 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:42.924462 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:42.925208 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:42.925212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:42.925212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:42.925420 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:42.925462 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:42.925478 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:42.925557 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:42.925761 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 16:36:42.925829 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260812 16:36:42.926994 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:42.932026 13152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:42.928481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36751 (local address 127.19.238.126:36187)
0812 16:36:42.928515 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:42.928516 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:42.928640 (+ 124us) server_negotiation.cc:407] Connection header received
0812 16:36:42.928682 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:42.928683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:42.928695 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:42.928754 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:42.928876 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:42.929641 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:42.930622 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:42.930965 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:42.931044 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:42.931751 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:42.931752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:42.931752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:42.931758 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:42.931773 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:42.931778 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:42.931865 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:42.931913 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 16:36:42.931940 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 16:36:42.960511 13162 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:42.961099 13162 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:42.967319 13162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6071e8a7b04b4951bbb8688a5ab170ea: Generated new TSK 0
I20260812 16:36:42.967550 13162 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:42.972652 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 13072
2026-08-12T16:36:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:43 dist-test-slave-mvvj krb5kdc[13182](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:43 dist-test-slave-mvvj krb5kdc[13182](info): set up 2 sockets
Aug 12 16:36:43 dist-test-slave-mvvj krb5kdc[13182](info): commencing operation
krb5kdc: starting...
W20260812 16:36:44.162173 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:44 dist-test-slave-mvvj krb5kdc[13182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:36:44.295213 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:41391
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:32859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:41391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:44.378659 13197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:44.378925 13197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:44.378964 13197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:44.380211 13197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:44.380285 13197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:44.380307 13197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:44.380329 13197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:44.380352 13197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:44.381939 13197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:32859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:41391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:41391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:44.382239 13197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:44.382498 13197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:44.387701 13202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:44.387704 13207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:44.387957 13203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:44.388211 13197 server_base.cc:1034] running on GCE node
I20260812 16:36:44.388446 13197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:44.388725 13197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:44.389880 13197 hybrid_clock.cc:648] HybridClock initialized: now 1786552604389864 us; error 33 us; skew 500 ppm
Aug 12 16:36:44 dist-test-slave-mvvj krb5kdc[13182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552604, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:44.392885 13197 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:36:44.393417 13197 webserver.cc:460] Webserver started at http://127.19.238.126:36307/ using document root <none> and password file <none>
I20260812 16:36:44.393720 13197 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:44.393795 13197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:44.393950 13197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:44.395175 13197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "ea4612df71674a9dae7c817c2de02c35"
format_stamp: "Formatted at 2026-08-12 16:36:44 on dist-test-slave-mvvj"
server_key: "2da621868773f5e47293a24546c65ab2"
server_key_iv: "c0bc8e9544dbab94f2678f90873ccb6a"
server_key_version: "encryptionkey@0"
I20260812 16:36:44.395583 13197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "ea4612df71674a9dae7c817c2de02c35"
format_stamp: "Formatted at 2026-08-12 16:36:44 on dist-test-slave-mvvj"
server_key: "2da621868773f5e47293a24546c65ab2"
server_key_iv: "c0bc8e9544dbab94f2678f90873ccb6a"
server_key_version: "encryptionkey@0"
I20260812 16:36:44.397152 13197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:36:44.398029 13214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:44.398277 13197 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:36:44.398404 13197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "ea4612df71674a9dae7c817c2de02c35"
format_stamp: "Formatted at 2026-08-12 16:36:44 on dist-test-slave-mvvj"
server_key: "2da621868773f5e47293a24546c65ab2"
server_key_iv: "c0bc8e9544dbab94f2678f90873ccb6a"
server_key_version: "encryptionkey@0"
I20260812 16:36:44.398527 13197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:44.402089 13197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:44.404541 13197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:44.404711 13197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:44.409019 13197 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:41391
I20260812 16:36:44.409053 13273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:41391 every 8 connection(s)
I20260812 16:36:44.409438 13197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:44.410506 13274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:44.411154 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 13197
I20260812 16:36:44.411283 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:44.411491 20409 external_mini_cluster.cc:1428] Setting key 078c0bacad59dfce58b9886f6cec7098
I20260812 16:36:44.414113 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:44.412122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50675 (local address 127.19.238.126:41391)
0812 16:36:44.412331 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:44.412339 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:44.412366 (+ 27us) server_negotiation.cc:407] Connection header received
0812 16:36:44.412782 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:44.412788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:44.412819 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:44.412939 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:44.413639 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:44.413642 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:44.413642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:44.413646 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:44.413685 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:44.413695 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:44.413753 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:44.413844 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 16:36:44.413899 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":92,"threads_started":1}
I20260812 16:36:44.414129 13274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35: Bootstrap starting.
I20260812 16:36:44.414909 13274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:44.415040 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:44.415266 13274 log.cc:824] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:44.416065 13274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35: No bootstrap required, opened a new log
I20260812 16:36:44.417591 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:44.416560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51351 (local address 127.19.238.126:41391)
0812 16:36:44.416597 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:44.416599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:44.416653 (+ 54us) server_negotiation.cc:407] Connection header received
0812 16:36:44.416726 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:44.416726 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:44.416730 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:44.416772 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:44.417340 (+ 568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:44.417341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:44.417341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:36:44.417345 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:44.417358 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:36:44.417361 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:44.417408 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:44.417481 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 16:36:44.417494 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 16:36:44.418578 13274 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4612df71674a9dae7c817c2de02c35" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 41391 } }
I20260812 16:36:44.418751 13274 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:36:44.418798 13274 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea4612df71674a9dae7c817c2de02c35, State: Initialized, Role: FOLLOWER
I20260812 16:36:44.418932 13274 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4612df71674a9dae7c817c2de02c35" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 41391 } }
I20260812 16:36:44.419032 13274 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:36:44.419077 13274 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:36:44.419130 13274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:44.419727 13274 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4612df71674a9dae7c817c2de02c35" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 41391 } }
I20260812 16:36:44.419878 13274 leader_election.cc:302] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea4612df71674a9dae7c817c2de02c35; no voters:
I20260812 16:36:44.420068 13274 leader_election.cc:288] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:36:44.420121 13285 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:44.420305 13285 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35 [term 1 LEADER]: Becoming Leader. State: Replica: ea4612df71674a9dae7c817c2de02c35, State: Running, Role: LEADER
I20260812 16:36:44.420331 13274 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:44.420509 13285 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4612df71674a9dae7c817c2de02c35" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 41391 } }
I20260812 16:36:44.420954 13286 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea4612df71674a9dae7c817c2de02c35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4612df71674a9dae7c817c2de02c35" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 41391 } } }
I20260812 16:36:44.421002 13287 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea4612df71674a9dae7c817c2de02c35. Latest consensus state: current_term: 1 leader_uuid: "ea4612df71674a9dae7c817c2de02c35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4612df71674a9dae7c817c2de02c35" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 41391 } } }
I20260812 16:36:44.421160 13287 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:44.421373 13286 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35 [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:44.421543 13296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:44.422062 13296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:44.423591 13296 catalog_manager.cc:1347] Generated new cluster ID: 4778a099e96d4370884e522b035dc9ab
I20260812 16:36:44.423666 13296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:44.435717 13296 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:44.435972 13296 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:44.438730 13296 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea4612df71674a9dae7c817c2de02c35: Generated new TSK 0
I20260812 16:36:44.438926 13296 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:36:44.449406 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 13197
2026-08-12T16:36:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1475 ms)
[----------] 18 tests from AuthTokenIssuingTest (25923 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:44 dist-test-slave-mvvj krb5kdc[13308](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:44 dist-test-slave-mvvj krb5kdc[13308](info): set up 2 sockets
Aug 12 16:36:44 dist-test-slave-mvvj krb5kdc[13308](info): commencing operation
krb5kdc: starting...
W20260812 16:36:45.564244 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:45 dist-test-slave-mvvj krb5kdc[13308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:36:45.695099 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:38159
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:39369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:38159,127.19.238.125:37309,127.19.238.124:34845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:45.778585 13325 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 16:36:45.778846 13325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:45.778888 13325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:45.778929 13325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:45.780186 13325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:45.780251 13325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:45.780277 13325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:45.780308 13325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:45.780334 13325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:45.781842 13325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:39369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:38159,127.19.238.125:37309,127.19.238.124:34845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:38159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:45.782140 13325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:45.782382 13325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:45.787225 13331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:45.787303 13330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:45.787261 13333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:45.787806 13325 server_base.cc:1034] running on GCE node
I20260812 16:36:45.788128 13325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:45.788437 13325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:45.789654 13325 hybrid_clock.cc:648] HybridClock initialized: now 1786552605789638 us; error 31 us; skew 500 ppm
Aug 12 16:36:45 dist-test-slave-mvvj krb5kdc[13308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552605, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:45.792292 13325 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:36:45.792785 13325 webserver.cc:460] Webserver started at http://127.19.238.126:44909/ using document root <none> and password file <none>
I20260812 16:36:45.793047 13325 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:45.793119 13325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:45.793267 13325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:45.794391 13325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "2d752656945c4c798312991f55f0a88b"
format_stamp: "Formatted at 2026-08-12 16:36:45 on dist-test-slave-mvvj"
server_key: "ad36cd869d98339ae41994b55ac92ad9"
server_key_iv: "a548c3dfaa16b35a64b2cd0eb58efe5d"
server_key_version: "encryptionkey@0"
I20260812 16:36:45.794770 13325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "2d752656945c4c798312991f55f0a88b"
format_stamp: "Formatted at 2026-08-12 16:36:45 on dist-test-slave-mvvj"
server_key: "ad36cd869d98339ae41994b55ac92ad9"
server_key_iv: "a548c3dfaa16b35a64b2cd0eb58efe5d"
server_key_version: "encryptionkey@0"
I20260812 16:36:45.796241 13325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:36:45.797073 13345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:45.797286 13325 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:36:45.797462 13325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "2d752656945c4c798312991f55f0a88b"
format_stamp: "Formatted at 2026-08-12 16:36:45 on dist-test-slave-mvvj"
server_key: "ad36cd869d98339ae41994b55ac92ad9"
server_key_iv: "a548c3dfaa16b35a64b2cd0eb58efe5d"
server_key_version: "encryptionkey@0"
I20260812 16:36:45.797549 13325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:45.806066 13325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:45.808195 13325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:45.808367 13325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:45.812378 13325 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:38159
I20260812 16:36:45.812453 13405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:38159 every 8 connection(s)
I20260812 16:36:45.812819 13325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:45.813758 13406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:45.815398 13406 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:36:45.820789 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 13325
I20260812 16:36:45.820936 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:45.821180 20409 external_mini_cluster.cc:1428] Setting key 871ce7acb7b219b0ce33be9f70e300f3
Aug 12 16:36:45 dist-test-slave-mvvj krb5kdc[13308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552605, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:36:45.826316 13409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:45.816238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48052 (local address 127.19.238.126:38159)
0812 16:36:45.816464 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:45.816468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:45.816505 (+ 37us) server_negotiation.cc:407] Connection header received
0812 16:36:45.816977 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:45.816979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:45.816997 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:45.817093 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:45.817722 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:45.818560 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:45.819392 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:45.819741 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:45.819823 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:45.822521 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:45.822529 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:45.822529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:45.822536 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:45.824575 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:45.825053 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:45.825056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:45.825056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:45.825134 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:45.825384 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:45.825384 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:45.825385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:45.825624 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:45.825688 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:45.825946 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 16:36:45.826043 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":106,"threads_started":1}
I20260812 16:36:45.826397 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:45.816147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:38159 (local address 127.0.0.1:48052)
0812 16:36:45.816327 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:45.816369 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:45.816828 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:45.817156 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:45.817158 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:45.817445 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:45.817605 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:45.817609 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:45.818647 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:45.818648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:45.819317 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:45.819319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:45.819879 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:45.819881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:45.819965 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:45.820508 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:45.820517 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:45.822410 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:45.824674 (+ 2264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:45.824682 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:45.824684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:45.824971 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:45.825202 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:45.825203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:45.825203 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:45.825320 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:45.825703 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:45.825711 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:45.825798 (+ 87us) client_negotiation.cc:769] Sending connection context
0812 16:36:45.825910 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 16:36:45.826158 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":69,"thread_start_us":94,"threads_started":1}
I20260812 16:36:45.827186 13406 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:36:45.827591 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:45.827435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:37309 (local address 127.0.0.1:55944)
0812 16:36:45.827500 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:45.827556 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.125:37309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 16:36:45.827764 13349 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.238.125:37309: connect: Connection refused (error 111)
I20260812 16:36:45.828251 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:45.828156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:37309 (local address 127.0.0.1:55960)
0812 16:36:45.828207 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:45.828233 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.125:37309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 16:36:45.828377 13406 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.125:37309: Network error: Client connection negotiation failed: client connection to 127.19.238.125:37309: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.19.238.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.125.keytab.
Entry for principal kudu/127.19.238.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.125.keytab.
I20260812 16:36:45.852155 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.125:37309
--webserver_interface=127.19.238.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.125.keytab
--principal=kudu/127.19.238.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:39369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:38159,127.19.238.125:37309,127.19.238.124:34845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:36:45.877612 13406 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } attempt: 1
I20260812 16:36:45.878108 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:45.877965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:37309 (local address 127.0.0.1:55970)
0812 16:36:45.878036 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:45.878077 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.125:37309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 16:36:45.878487 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:45.878385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:37309 (local address 127.0.0.1:55986)
0812 16:36:45.878434 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:45.878462 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.125:37309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 16:36:45.878588 13406 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.125:37309: Network error: Client connection negotiation failed: client connection to 127.19.238.125:37309: connect: Connection refused (error 111)
W20260812 16:36:45.938400 13415 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 16:36:45.938685 13415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:45.938717 13415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:45.938764 13415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:45.940062 13415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:45.940130 13415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:45.940155 13415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:45.940187 13415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:45.940219 13415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:45.941812 13415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:39369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:38159,127.19.238.125:37309,127.19.238.124:34845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.125.keytab
--principal=kudu/127.19.238.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.125:37309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.238.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:45.942116 13415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:45.942358 13415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 16:36:45.946815 13406 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } attempt: 2
I20260812 16:36:45.947340 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:45.947144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:37309 (local address 127.0.0.1:55992)
0812 16:36:45.947251 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:45.947281 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.125:37309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 16:36:45.947656 13415 server_base.cc:1034] running on GCE node
I20260812 16:36:45.947714 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:45.947607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:37309 (local address 127.0.0.1:56008)
0812 16:36:45.947659 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:45.947686 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.125:37309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 16:36:45.947805 13406 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.125:37309: Network error: Client connection negotiation failed: client connection to 127.19.238.125:37309: connect: Connection refused (error 111)
W20260812 16:36:45.947777 13423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:45.947781 13420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:45.947942 13421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:45.948172 13415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:45.948450 13415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:45.949604 13415 hybrid_clock.cc:648] HybridClock initialized: now 1786552605949583 us; error 47 us; skew 500 ppm
Aug 12 16:36:45 dist-test-slave-mvvj krb5kdc[13308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552605, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:45.952364 13415 init.cc:377] Logged in from keytab as kudu/127.19.238.125@KRBTEST.COM (short username kudu)
I20260812 16:36:45.952913 13415 webserver.cc:460] Webserver started at http://127.19.238.125:41797/ using document root <none> and password file <none>
I20260812 16:36:45.953157 13415 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:45.953229 13415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:45.953390 13415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:45.954563 13415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-1/data/instance:
uuid: "019112ee194148dca14b1596c3269eab"
format_stamp: "Formatted at 2026-08-12 16:36:45 on dist-test-slave-mvvj"
server_key: "3e5f673337aa9d55fae666465509ae7f"
server_key_iv: "119afd03db71602069c8ba92bdda5346"
server_key_version: "encryptionkey@0"
I20260812 16:36:45.954957 13415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-1/wal/instance:
uuid: "019112ee194148dca14b1596c3269eab"
format_stamp: "Formatted at 2026-08-12 16:36:45 on dist-test-slave-mvvj"
server_key: "3e5f673337aa9d55fae666465509ae7f"
server_key_iv: "119afd03db71602069c8ba92bdda5346"
server_key_version: "encryptionkey@0"
I20260812 16:36:45.956488 13415 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 16:36:45.957311 13431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:45.957543 13415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:45.957652 13415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-1/wal
uuid: "019112ee194148dca14b1596c3269eab"
format_stamp: "Formatted at 2026-08-12 16:36:45 on dist-test-slave-mvvj"
server_key: "3e5f673337aa9d55fae666465509ae7f"
server_key_iv: "119afd03db71602069c8ba92bdda5346"
server_key_version: "encryptionkey@0"
I20260812 16:36:45.957736 13415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:45.982493 13415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:45.984933 13415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:45.985110 13415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:45.989614 13415 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.125:37309
I20260812 16:36:45.989655 13488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.125:37309 every 8 connection(s)
I20260812 16:36:45.990057 13415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-1/data/info.pb
I20260812 16:36:45.990965 13489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:45.992607 13489 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:36:45 dist-test-slave-mvvj krb5kdc[13308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552605, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.125@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:36:45.998422 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 13415
I20260812 16:36:45.998531 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-1/wal/instance
I20260812 16:36:45.998754 20409 external_mini_cluster.cc:1428] Setting key 14754d191d80b77fd0cc4c6c7f238455
I20260812 16:36:46.002473 13409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:45.993398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48058 (local address 127.19.238.126:38159)
0812 16:36:45.993515 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:45.993517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:45.993696 (+ 179us) server_negotiation.cc:407] Connection header received
0812 16:36:45.994169 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:45.994170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:45.994182 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:45.994242 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:45.994845 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:45.995342 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:45.996426 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:45.996639 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:45.996698 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:45.998937 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:45.998940 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:45.998940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:45.998948 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:46.000854 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.001405 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.001406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.001407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.001498 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.001806 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.001807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.001807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.001915 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:46.001970 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:46.002305 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 16:36:46.002373 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 16:36:46.002828 13491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:45.993377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:38159 (local address 127.0.0.1:48058)
0812 16:36:45.993562 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:45.993604 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:45.994082 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:45.994296 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:45.994299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:45.994565 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:45.994762 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:45.994766 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:45.995472 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:45.995473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:45.996339 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:45.996342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:45.996756 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:45.996757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:45.996874 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:45.997345 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:45.997354 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:45.998846 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:46.000948 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:46.000956 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:46.000958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:46.001312 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:46.001572 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:46.001573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:46.001573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:46.001739 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:46.001984 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:46.001991 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:46.002169 (+ 178us) client_negotiation.cc:769] Sending connection context
0812 16:36:46.002286 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 16:36:46.002543 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260812 16:36:46.003448 13489 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:36:46 dist-test-slave-mvvj krb5kdc[13308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552605, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.125@KRBTEST.COM for kudu/127.19.238.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.238.124@KRBTEST.COM; defaulting to no policy
I20260812 16:36:46.011461 13491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.003751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:37309 (local address 127.0.0.1:56016)
0812 16:36:46.003811 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:46.003825 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:46.003906 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:46.004128 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:46.004129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:46.004286 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:46.004360 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:46.004362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.005224 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.005226 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:46.006134 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:46.006137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.006608 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.006609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:46.006691 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.007078 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:46.007083 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:46.008512 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:46.010229 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:46.010230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:46.010231 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:46.010395 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:46.010625 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:46.010626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:46.010626 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:46.010687 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:46.011045 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:46.011046 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:46.011086 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 16:36:46.011177 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 16:36:46.011285 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 16:36:46.011498 13494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.003803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56016 (local address 127.19.238.125:37309)
0812 16:36:46.003991 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.003994 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.004009 (+ 15us) server_negotiation.cc:407] Connection header received
0812 16:36:46.004029 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.004030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.004041 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.004103 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.004435 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.005129 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.006219 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.006527 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.006583 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.008607 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:46.008622 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:46.008622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:46.008630 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:46.010128 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.010495 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.010498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.010498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.010552 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.010748 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.010749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.010750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.010949 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:46.011046 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:46.011214 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 16:36:46.011340 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":153,"thread_start_us":107,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.124@KRBTEST.COM" created.
I20260812 16:36:46.012509 13489 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:36:46.012919 13491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.012772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:34845 (local address 127.0.0.1:43092)
0812 16:36:46.012826 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:46.012880 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:34845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 16:36:46.013206 13435 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.238.124:34845: connect: Connection refused (error 111)
I20260812 16:36:46.013747 13491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.013633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:34845 (local address 127.0.0.1:43094)
0812 16:36:46.013684 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:46.013715 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:34845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 16:36:46.013912 13489 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.124:34845: Network error: Client connection negotiation failed: client connection to 127.19.238.124:34845: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.124.keytab.
Entry for principal kudu/127.19.238.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.124.keytab.
I20260812 16:36:46.031584 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.124:34845
--webserver_interface=127.19.238.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.124.keytab
--principal=kudu/127.19.238.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:39369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:38159,127.19.238.125:37309,127.19.238.124:34845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:36:46.038985 13406 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } attempt: 3
Aug 12 16:36:46 dist-test-slave-mvvj krb5kdc[13308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552605, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for kudu/127.19.238.125@KRBTEST.COM
I20260812 16:36:46.046352 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.039338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:37309 (local address 127.0.0.1:56024)
0812 16:36:46.039401 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:46.039420 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:46.039523 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:46.039675 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:46.039676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:46.039894 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:46.039965 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:46.039968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.040649 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.040650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:46.041173 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:46.041174 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.041579 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.041580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:46.041663 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.042144 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:46.042149 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:46.043393 (+ 1244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:46.045283 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:46.045284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:46.045284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:46.045557 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:46.045765 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:46.045765 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:46.045766 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:46.045823 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:46.046097 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:46.046098 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:46.046140 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 16:36:46.046197 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 16:36:46.046279 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 16:36:46.046425 13494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.039452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56024 (local address 127.19.238.125:37309)
0812 16:36:46.039534 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.039537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.039551 (+ 14us) server_negotiation.cc:407] Connection header received
0812 16:36:46.039585 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.039586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.039596 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.039670 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.040063 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.040561 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.041269 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.041490 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.041577 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.043486 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:46.043487 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:46.043488 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:46.043497 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:46.045188 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.045632 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.045633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.045633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.045693 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.045886 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.045887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.045887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.046029 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:46.046094 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:46.046238 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 16:36:46.046327 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 16:36:46.046787 13406 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:36:46.047153 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.047039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:34845 (local address 127.0.0.1:43098)
0812 16:36:46.047094 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:46.047129 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:34845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 16:36:46.047451 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.047355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:34845 (local address 127.0.0.1:43102)
0812 16:36:46.047408 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:46.047433 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:34845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 16:36:46.047552 13406 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.124:34845: Network error: Client connection negotiation failed: client connection to 127.19.238.124:34845: connect: Connection refused (error 111)
I20260812 16:36:46.063174 13489 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } attempt: 1
I20260812 16:36:46.063724 13491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.063584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:34845 (local address 127.0.0.1:43108)
0812 16:36:46.063649 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:46.063692 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:34845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 16:36:46.064159 13491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.064069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:34845 (local address 127.0.0.1:43114)
0812 16:36:46.064114 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:46.064141 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:34845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 16:36:46.064356 13489 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.124:34845: Network error: Client connection negotiation failed: client connection to 127.19.238.124:34845: connect: Connection refused (error 111)
I20260812 16:36:46.078765 13406 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } attempt: 1
I20260812 16:36:46.079313 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.079137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:34845 (local address 127.0.0.1:43130)
0812 16:36:46.079215 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:46.079270 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:34845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 16:36:46.079730 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.079623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:34845 (local address 127.0.0.1:43136)
0812 16:36:46.079674 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:46.079706 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:34845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 16:36:46.079922 13406 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.124:34845: Network error: Client connection negotiation failed: client connection to 127.19.238.124:34845: connect: Connection refused (error 111)
W20260812 16:36:46.120636 13497 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 16:36:46.120864 13497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:46.120882 13497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:46.120903 13497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:46.122112 13497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:46.122153 13497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:46.122219 13497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:46.122243 13497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:46.122267 13497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:46.123814 13497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:39369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:38159,127.19.238.125:37309,127.19.238.124:34845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.124.keytab
--principal=kudu/127.19.238.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.124:34845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.238.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:46.124145 13497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:46.124394 13497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:46.129545 13504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:46.129590 13503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:46.129545 13507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:46.129868 13497 server_base.cc:1034] running on GCE node
I20260812 16:36:46.130133 13497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:46.130489 13497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:46.131690 13497 hybrid_clock.cc:648] HybridClock initialized: now 1786552606131677 us; error 54 us; skew 500 ppm
I20260812 16:36:46.132517 13489 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } attempt: 2
I20260812 16:36:46.132994 13491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.132841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:34845 (local address 127.0.0.1:43142)
0812 16:36:46.132908 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:46.132957 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:34845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 16:36:46.133399 13491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.133293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:34845 (local address 127.0.0.1:43152)
0812 16:36:46.133343 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:46.133373 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:34845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 16:36:46.133522 13489 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.124:34845: Network error: Client connection negotiation failed: client connection to 127.19.238.124:34845: connect: Connection refused (error 111)
Aug 12 16:36:46 dist-test-slave-mvvj krb5kdc[13308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552606, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:46.134611 13497 init.cc:377] Logged in from keytab as kudu/127.19.238.124@KRBTEST.COM (short username kudu)
I20260812 16:36:46.135069 13497 webserver.cc:460] Webserver started at http://127.19.238.124:41627/ using document root <none> and password file <none>
I20260812 16:36:46.135304 13497 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:46.135375 13497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:46.135516 13497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:46.136669 13497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-2/data/instance:
uuid: "c5ee9273031b49abbc8ca70234f51ce1"
format_stamp: "Formatted at 2026-08-12 16:36:46 on dist-test-slave-mvvj"
server_key: "583b44c106458d2b3799e7224c2d2371"
server_key_iv: "1f2cc12cfda9b2b1ac165ff0aff9ee62"
server_key_version: "encryptionkey@0"
I20260812 16:36:46.137066 13497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-2/wal/instance:
uuid: "c5ee9273031b49abbc8ca70234f51ce1"
format_stamp: "Formatted at 2026-08-12 16:36:46 on dist-test-slave-mvvj"
server_key: "583b44c106458d2b3799e7224c2d2371"
server_key_iv: "1f2cc12cfda9b2b1ac165ff0aff9ee62"
server_key_version: "encryptionkey@0"
I20260812 16:36:46.138520 13497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 16:36:46.139287 13515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:46.139505 13497 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 16:36:46.139616 13497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-2/wal
uuid: "c5ee9273031b49abbc8ca70234f51ce1"
format_stamp: "Formatted at 2026-08-12 16:36:46 on dist-test-slave-mvvj"
server_key: "583b44c106458d2b3799e7224c2d2371"
server_key_iv: "1f2cc12cfda9b2b1ac165ff0aff9ee62"
server_key_version: "encryptionkey@0"
I20260812 16:36:46.139693 13497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:46.147889 13497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:46.150005 13497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:46.150187 13497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:46.154562 13497 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.124:34845
I20260812 16:36:46.154620 13573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.124:34845 every 8 connection(s)
I20260812 16:36:46.154986 13497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-2/data/info.pb
I20260812 16:36:46.155107 13406 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } attempt: 2
I20260812 16:36:46.156296 13574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:46.157294 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 13497
I20260812 16:36:46.157408 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786552327177731-20409-0/minicluster-data/master-2/wal/instance
I20260812 16:36:46.157620 20409 external_mini_cluster.cc:1428] Setting key 72116eeb2c6fa7011db3cd086607095b
I20260812 16:36:46.158757 13574 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:36:46 dist-test-slave-mvvj krb5kdc[13308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552605, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for kudu/127.19.238.124@KRBTEST.COM
Aug 12 16:36:46 dist-test-slave-mvvj krb5kdc[13308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552606, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.124@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
Aug 12 16:36:46 dist-test-slave-mvvj krb5kdc[13308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.124@KRBTEST.COM
I20260812 16:36:46.166129 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.155455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:34845 (local address 127.0.0.1:43168)
0812 16:36:46.155522 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:46.155535 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:46.155600 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:46.156508 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:46.156510 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:46.156701 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:46.156754 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:46.156756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.158134 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.158136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:46.158935 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:46.158938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.159806 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.159807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:46.159949 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.160413 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:46.160418 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:46.161880 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:46.164702 (+ 2822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:46.164703 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:46.164704 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:46.164940 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:46.165249 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:46.165250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:46.165250 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:46.165323 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:46.165845 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:46.165847 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:46.165896 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 16:36:46.165968 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 16:36:46.166059 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 16:36:46.166489 13576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.155589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43168 (local address 127.19.238.124:34845)
0812 16:36:46.155777 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.155785 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.155810 (+ 25us) server_negotiation.cc:407] Connection header received
0812 16:36:46.156298 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.156304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.156356 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.156513 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.156861 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.158036 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.159053 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.159689 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.160170 (+ 481us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.162142 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:46.162167 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:46.162168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:46.162182 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:46.164615 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.165027 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.165031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.165031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.165138 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.165408 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.165410 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.165410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.165747 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:46.165862 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:46.166052 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 16:36:46.166190 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":93,"threads_started":1}
I20260812 16:36:46.168620 13409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.159476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48064 (local address 127.19.238.126:38159)
0812 16:36:46.159537 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.159539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.159741 (+ 202us) server_negotiation.cc:407] Connection header received
0812 16:36:46.159785 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.159786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.159797 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.159914 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.160502 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.161100 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.162187 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.162498 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.162569 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.164883 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:46.164884 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:46.164885 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:46.164892 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:46.166907 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.167680 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.167681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.167681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.167740 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.168022 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.168023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.168024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.168183 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:46.168255 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:46.168460 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 16:36:46.168531 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:36:46.168638 13580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.159414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:38159 (local address 127.0.0.1:48064)
0812 16:36:46.159617 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:46.159663 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:46.159735 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:46.159942 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:46.159945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:46.160334 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:46.160437 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:46.160440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.161184 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.161185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:46.162100 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:46.162103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.162579 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.162580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:46.162676 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.163168 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:46.163173 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:46.164770 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:46.166984 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:46.166991 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:46.166993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:46.167364 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:46.167868 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:46.167869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:46.167869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:46.167940 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:46.168265 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:46.168273 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:46.168327 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 16:36:46.168417 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 16:36:46.168531 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":77,"threads_started":1}
I20260812 16:36:46.169426 13406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b: Bootstrap starting.
I20260812 16:36:46.169705 13578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.158012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43176 (local address 127.19.238.124:34845)
0812 16:36:46.159292 (+ 1280us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.159299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.159305 (+ 6us) server_negotiation.cc:407] Connection header received
0812 16:36:46.159326 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.159327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.159341 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.159430 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.160469 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.161265 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.162335 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.162582 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.162650 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.165936 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:46.165938 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:46.165938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:46.165944 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:46.168295 (+ 2351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.168955 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.168956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.168957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.169009 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.169217 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.169218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.169218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.169298 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:46.169395 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:46.169544 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 16:36:46.169624 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1259,"thread_start_us":112,"threads_started":1}
I20260812 16:36:46.170195 13406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:46.170493 13406 log.cc:824] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:46.171265 13574 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:36:46.171550 13406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b: No bootstrap required, opened a new log
I20260812 16:36:46.173373 13406 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d752656945c4c798312991f55f0a88b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } } peers { permanent_uuid: "019112ee194148dca14b1596c3269eab" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } } peers { permanent_uuid: "c5ee9273031b49abbc8ca70234f51ce1" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } }
I20260812 16:36:46.173522 13406 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d752656945c4c798312991f55f0a88b, State: Initialized, Role: FOLLOWER
I20260812 16:36:46.173595 13406 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d752656945c4c798312991f55f0a88b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } } peers { permanent_uuid: "019112ee194148dca14b1596c3269eab" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } } peers { permanent_uuid: "c5ee9273031b49abbc8ca70234f51ce1" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } }
I20260812 16:36:46.173930 13406 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:46.175984 13582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d752656945c4c798312991f55f0a88b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } } peers { permanent_uuid: "019112ee194148dca14b1596c3269eab" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } } peers { permanent_uuid: "c5ee9273031b49abbc8ca70234f51ce1" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } } }
I20260812 16:36:46.176092 13582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [sys.catalog]: This master's current role is: FOLLOWER
Aug 12 16:36:46 dist-test-slave-mvvj krb5kdc[13308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552606, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.124@KRBTEST.COM for kudu/127.19.238.125@KRBTEST.COM
W20260812 16:36:46.176679 13596 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:36:46.176743 13596 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 12 16:36:46 dist-test-slave-mvvj krb5kdc[13308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:36:46.177382 13325 master_runner.cc:186] Error getting master registration for 127.19.238.125:37309: OK, instance_id { permanent_uuid: "019112ee194148dca14b1596c3269eab" instance_seqno: 1786552605988291 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 16:36:46.180955 13325 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 16:36:46.180951 13580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.171656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:37309 (local address 127.0.0.1:56032)
0812 16:36:46.171758 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:46.171768 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:46.171806 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:46.171986 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:46.171987 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:46.172151 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:46.172194 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:46.172196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.173150 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.173151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:46.173892 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:46.173895 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.174377 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.174378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:46.174462 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.175141 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:46.175146 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:46.177754 (+ 2608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:46.179785 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:46.179786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:46.179787 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:46.180032 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:46.180276 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:46.180277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:46.180277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:46.180350 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:46.180638 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:46.180640 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:46.180685 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 16:36:46.180737 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 16:36:46.180850 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260812 16:36:46.181345 13494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.171790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56032 (local address 127.19.238.125:37309)
0812 16:36:46.171862 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.171864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.171872 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:36:46.171898 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.171899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.171912 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.171971 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.172287 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.173057 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.173987 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.174308 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.174374 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.177852 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:46.177854 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:46.177855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:46.177864 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:46.179688 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.180139 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.180140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.180140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.180204 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.180418 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.180418 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.180419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.180554 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:46.180612 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:46.181190 (+ 578us) server_negotiation.cc:299] Negotiation successful
0812 16:36:46.181276 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 16:36:46.181771 13409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.171616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48074 (local address 127.19.238.126:38159)
0812 16:36:46.171653 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.171655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.171797 (+ 142us) server_negotiation.cc:407] Connection header received
0812 16:36:46.171944 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.171945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.171955 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.171985 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.172629 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.173272 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.174136 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.174420 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.174459 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.177500 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:46.177502 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:46.177502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:46.177509 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:46.179732 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.180169 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.180171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.180171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.180231 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.180455 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.180456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.180457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.180591 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:46.180751 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:46.181593 (+ 842us) server_negotiation.cc:299] Negotiation successful
0812 16:36:46.181679 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:36:46.181879 13574 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:36:46 dist-test-slave-mvvj krb5kdc[13308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552606, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.124@KRBTEST.COM for kudu/127.19.238.124@KRBTEST.COM
Aug 12 16:36:46 dist-test-slave-mvvj krb5kdc[13308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.125@KRBTEST.COM
I20260812 16:36:46.189438 13580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.182136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:34845 (local address 127.0.0.1:43186)
0812 16:36:46.182171 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:46.182182 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:46.182221 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:46.182385 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:46.182386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:46.182537 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:46.182576 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:46.182577 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.183489 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.183490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:46.184174 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:46.184177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.184563 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.184564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:46.184611 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.185012 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:46.185016 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:46.186328 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:46.188125 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:46.188126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:46.188127 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:46.188295 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:46.188621 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:46.188622 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:46.188622 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:46.188673 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:46.189038 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:46.189039 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:46.189080 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 16:36:46.189249 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 16:36:46.189372 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260812 16:36:46.189790 13578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.182171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43186 (local address 127.19.238.124:34845)
0812 16:36:46.182209 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.182211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.182219 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:36:46.182274 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.182275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.182286 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.182350 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.182652 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.183400 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.184269 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.184480 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.184564 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.186441 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:46.186442 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:46.186442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:46.186450 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:46.188033 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.188369 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.188370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.188370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.188431 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.188747 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.188748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.188748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.188946 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:46.189014 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:46.189640 (+ 626us) server_negotiation.cc:299] Negotiation successful
0812 16:36:46.189712 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 16:36:46.191123 13494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.182378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56046 (local address 127.19.238.125:37309)
0812 16:36:46.182409 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.182411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.182435 (+ 24us) server_negotiation.cc:407] Connection header received
0812 16:36:46.182506 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.182507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.182517 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.182570 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.183093 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.183615 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.184621 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.185217 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.185290 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.188149 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:46.188151 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:46.188151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:46.188159 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:46.189911 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.190304 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.190306 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.190306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.190377 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.190609 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.190610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.190610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.190743 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:46.190918 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:46.190962 (+ 44us) server_negotiation.cc:299] Negotiation successful
0812 16:36:46.191040 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 16:36:46.191430 13574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1: Bootstrap starting.
I20260812 16:36:46.192133 13574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:46.192476 13574 log.cc:824] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:46.193349 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:46.193413 13574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1: No bootstrap required, opened a new log
I20260812 16:36:46.194373 13375 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2d752656945c4c798312991f55f0a88b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48074
I20260812 16:36:46.194545 13375 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 16:36:46.194594 13375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:46.195609 13375 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d752656945c4c798312991f55f0a88b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } } peers { permanent_uuid: "019112ee194148dca14b1596c3269eab" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } } peers { permanent_uuid: "c5ee9273031b49abbc8ca70234f51ce1" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } }
I20260812 16:36:46.195654 13574 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d752656945c4c798312991f55f0a88b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } } peers { permanent_uuid: "019112ee194148dca14b1596c3269eab" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } } peers { permanent_uuid: "c5ee9273031b49abbc8ca70234f51ce1" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } }
I20260812 16:36:46.195791 13574 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5ee9273031b49abbc8ca70234f51ce1, State: Initialized, Role: FOLLOWER
I20260812 16:36:46.195951 13574 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d752656945c4c798312991f55f0a88b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } } peers { permanent_uuid: "019112ee194148dca14b1596c3269eab" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } } peers { permanent_uuid: "c5ee9273031b49abbc8ca70234f51ce1" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } }
I20260812 16:36:46.196169 13375 leader_election.cc:288] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [CANDIDATE]: Term 1 election: Requested vote from peers 019112ee194148dca14b1596c3269eab (127.19.238.125:37309), c5ee9273031b49abbc8ca70234f51ce1 (127.19.238.124:34845)
I20260812 16:36:46.196310 13574 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:46.196683 13545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d752656945c4c798312991f55f0a88b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c5ee9273031b49abbc8ca70234f51ce1"
I20260812 16:36:46.196794 13545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:46.198091 13545 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d752656945c4c798312991f55f0a88b in term 1.
I20260812 16:36:46.198444 13349 leader_election.cc:302] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d752656945c4c798312991f55f0a88b, c5ee9273031b49abbc8ca70234f51ce1; no voters:
I20260812 16:36:46.198580 13582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:46.198742 13582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [term 1 LEADER]: Becoming Leader. State: Replica: 2d752656945c4c798312991f55f0a88b, State: Running, Role: LEADER
I20260812 16:36:46.198853 13582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d752656945c4c798312991f55f0a88b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } } peers { permanent_uuid: "019112ee194148dca14b1596c3269eab" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } } peers { permanent_uuid: "c5ee9273031b49abbc8ca70234f51ce1" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } }
I20260812 16:36:46.199337 13582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d752656945c4c798312991f55f0a88b. Latest consensus state: current_term: 1 leader_uuid: "2d752656945c4c798312991f55f0a88b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d752656945c4c798312991f55f0a88b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } } peers { permanent_uuid: "019112ee194148dca14b1596c3269eab" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } } peers { permanent_uuid: "c5ee9273031b49abbc8ca70234f51ce1" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } } }
I20260812 16:36:46.199424 13582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [sys.catalog]: This master's current role is: LEADER
W20260812 16:36:46.199249 13617 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:36:46.199765 13617 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:36:46.199357 13599 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d752656945c4c798312991f55f0a88b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } } peers { permanent_uuid: "019112ee194148dca14b1596c3269eab" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } } peers { permanent_uuid: "c5ee9273031b49abbc8ca70234f51ce1" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } } }
I20260812 16:36:46.199904 13599 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:36:46.201591 13497 master_runner.cc:186] Error getting master registration for 127.19.238.125:37309: OK, instance_id { permanent_uuid: "019112ee194148dca14b1596c3269eab" instance_seqno: 1786552605988291 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 16:36:46.203815 13497 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 16:36:46.209514 13578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.198540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43196 (local address 127.19.238.124:34845)
0812 16:36:46.199563 (+ 1023us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.199566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.200863 (+ 1297us) server_negotiation.cc:407] Connection header received
0812 16:36:46.200960 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.200961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.200972 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.201002 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.201772 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.202491 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.203533 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.203803 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.203878 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.205923 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:46.205924 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:46.205925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:46.205932 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:46.208112 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.208539 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.208540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.208540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.208598 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.208903 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.208904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.208904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.209040 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:46.209112 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:46.209333 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 16:36:46.209418 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1001}
I20260812 16:36:46.210297 13409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.198126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.19.238.126:38159)
0812 16:36:46.198255 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.198257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.200684 (+ 2427us) server_negotiation.cc:407] Connection header received
0812 16:36:46.200774 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.200775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.200786 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.200842 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.202040 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.202783 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.204405 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.204718 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.204757 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.206895 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:46.206897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:46.206897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:46.206904 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:46.208550 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.209158 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.209159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.209159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.209229 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.209555 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.209556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.209556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.209686 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:46.209780 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:46.210105 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 16:36:46.210200 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260812 16:36:46.212414 13622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:46.212642 13622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:46.215057 13545 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1 [term 1 FOLLOWER]: Refusing update from remote peer 2d752656945c4c798312991f55f0a88b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:36:46.215504 13618 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5ee9273031b49abbc8ca70234f51ce1" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:36:46.217061 13494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.198280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56048 (local address 127.19.238.125:37309)
0812 16:36:46.198345 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.198348 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.200524 (+ 2176us) server_negotiation.cc:407] Connection header received
0812 16:36:46.200845 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.200847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.200859 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.200914 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.204801 (+ 3887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.205560 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.210120 (+ 4560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.211095 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.211145 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.213468 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:46.213470 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:46.213470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:46.213483 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:46.215292 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.215948 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.215949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.215949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.216017 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.216297 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.216298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.216298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.216460 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:46.216557 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:46.216872 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 16:36:46.216963 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 16:36:46.217043 13599 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d752656945c4c798312991f55f0a88b. Latest consensus state: current_term: 1 leader_uuid: "2d752656945c4c798312991f55f0a88b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d752656945c4c798312991f55f0a88b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } } peers { permanent_uuid: "019112ee194148dca14b1596c3269eab" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } } peers { permanent_uuid: "c5ee9273031b49abbc8ca70234f51ce1" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } } }
I20260812 16:36:46.217545 13599 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:36:46.219035 13582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d752656945c4c798312991f55f0a88b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d752656945c4c798312991f55f0a88b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } } peers { permanent_uuid: "019112ee194148dca14b1596c3269eab" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } } peers { permanent_uuid: "c5ee9273031b49abbc8ca70234f51ce1" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } } }
I20260812 16:36:46.219136 13582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:46.219300 13599 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d752656945c4c798312991f55f0a88b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d752656945c4c798312991f55f0a88b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } } peers { permanent_uuid: "019112ee194148dca14b1596c3269eab" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } } peers { permanent_uuid: "c5ee9273031b49abbc8ca70234f51ce1" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } } }
I20260812 16:36:46.219386 13599 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:36:46.219583 13627 mvcc.cc:205] Tried to move back new op lower bound from 7317719475050958848 to 7317719474991333376. Current Snapshot: MvccSnapshot[applied={T|T < 7317719475050958848}]
I20260812 16:36:46.220093 13630 mvcc.cc:205] Tried to move back new op lower bound from 7317719475050958848 to 7317719474991333376. Current Snapshot: MvccSnapshot[applied={T|T < 7317719475050958848}]
I20260812 16:36:46.224309 13622 catalog_manager.cc:1347] Generated new cluster ID: 79e43f431f4046f59818fed225ec57e7
I20260812 16:36:46.224383 13622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:46.224687 13489 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } attempt: 3
Aug 12 16:36:46 dist-test-slave-mvvj krb5kdc[13308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552605, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.125@KRBTEST.COM for kudu/127.19.238.124@KRBTEST.COM
I20260812 16:36:46.234263 13491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.224974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:34845 (local address 127.0.0.1:43200)
0812 16:36:46.225083 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:46.225095 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:46.225136 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:46.225384 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:46.225385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:46.225580 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:46.225627 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:46.225629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.226616 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.226617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:46.227461 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:46.227462 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.227993 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.227994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:46.228050 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.228560 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:46.228565 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:46.231175 (+ 2610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:46.233161 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:46.233163 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:46.233163 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:46.233382 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:46.233585 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:46.233585 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:46.233586 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:46.233642 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:46.233938 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:46.233939 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:46.233986 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 16:36:46.234049 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 16:36:46.234152 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260812 16:36:46.234802 13578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:46.225208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43200 (local address 127.19.238.124:34845)
0812 16:36:46.225252 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.225254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.225261 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:36:46.225283 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.225284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.225293 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.225338 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.225707 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.226514 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.227554 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:46.227872 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:46.227935 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:46.231282 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:46.231284 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:46.231284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:46.231291 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:46.233059 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.233462 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.233463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.233463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.233516 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:46.233705 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:46.233705 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:46.233706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:46.233848 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:46.233908 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:46.234619 (+ 711us) server_negotiation.cc:299] Negotiation successful
0812 16:36:46.234706 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:36:46.236621 13489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab: Bootstrap starting.
I20260812 16:36:46.237412 13489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:46.237721 13489 log.cc:824] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:46.239606 13489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab: No bootstrap required, opened a new log
I20260812 16:36:46.241736 13489 raft_consensus.cc:348] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d752656945c4c798312991f55f0a88b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } } peers { permanent_uuid: "019112ee194148dca14b1596c3269eab" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } } peers { permanent_uuid: "c5ee9273031b49abbc8ca70234f51ce1" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } }
I20260812 16:36:46.241925 13489 raft_consensus.cc:729] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019112ee194148dca14b1596c3269eab, State: Initialized, Role: FOLLOWER
I20260812 16:36:46.242019 13489 consensus_queue.cc:260] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d752656945c4c798312991f55f0a88b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } } peers { permanent_uuid: "019112ee194148dca14b1596c3269eab" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } } peers { permanent_uuid: "c5ee9273031b49abbc8ca70234f51ce1" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } }
I20260812 16:36:46.242527 13489 sys_catalog.cc:564] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:46.242597 13459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d752656945c4c798312991f55f0a88b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "019112ee194148dca14b1596c3269eab"
I20260812 16:36:46.242729 13459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:46.243697 13459 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d752656945c4c798312991f55f0a88b in term 1.
I20260812 16:36:46.243949 13458 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab [term 1 FOLLOWER]: Refusing update from remote peer 2d752656945c4c798312991f55f0a88b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:36:46.244908 13618 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [LEADER]: Connected to new peer: Peer: permanent_uuid: "019112ee194148dca14b1596c3269eab" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 16:36:46.247289 13652 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:36:46.247350 13652 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:36:46.247717 13636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d752656945c4c798312991f55f0a88b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d752656945c4c798312991f55f0a88b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } } peers { permanent_uuid: "019112ee194148dca14b1596c3269eab" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } } peers { permanent_uuid: "c5ee9273031b49abbc8ca70234f51ce1" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } } }
I20260812 16:36:46.247794 13636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:36:46.247911 13636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d752656945c4c798312991f55f0a88b. Latest consensus state: current_term: 1 leader_uuid: "2d752656945c4c798312991f55f0a88b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d752656945c4c798312991f55f0a88b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } } peers { permanent_uuid: "019112ee194148dca14b1596c3269eab" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } } peers { permanent_uuid: "c5ee9273031b49abbc8ca70234f51ce1" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } } }
I20260812 16:36:46.248232 13636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:36:46.248271 13636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d752656945c4c798312991f55f0a88b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d752656945c4c798312991f55f0a88b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } } peers { permanent_uuid: "019112ee194148dca14b1596c3269eab" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } } peers { permanent_uuid: "c5ee9273031b49abbc8ca70234f51ce1" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } } }
I20260812 16:36:46.248314 13636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:36:46.251925 13622 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:46.252764 13622 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:46.256389 13649 mvcc.cc:205] Tried to move back new op lower bound from 7317719475050958848 to 7317719474991333376. Current Snapshot: MvccSnapshot[applied={T|T < 7317719475050958848}]
I20260812 16:36:46.257333 13582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d752656945c4c798312991f55f0a88b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d752656945c4c798312991f55f0a88b" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 38159 } } peers { permanent_uuid: "019112ee194148dca14b1596c3269eab" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 37309 } } peers { permanent_uuid: "c5ee9273031b49abbc8ca70234f51ce1" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 34845 } } }
I20260812 16:36:46.257438 13582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:46.259936 13622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d752656945c4c798312991f55f0a88b: Generated new TSK 0
I20260812 16:36:46.260174 13622 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 16:36:46.291742 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:46.290002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48898 (local address 127.19.238.125:37309)
0812 16:36:46.290053 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.290055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.290995 (+ 940us) server_negotiation.cc:407] Connection header received
0812 16:36:46.291105 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.291106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.291118 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.291150 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.291658 (+ 508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48898: BlockingRecv error: recv got EOF from 127.0.0.1:48898 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 16:36:46.292701 13578 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:46.290399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43816 (local address 127.19.238.124:34845)
0812 16:36:46.290443 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.290445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.291979 (+ 1534us) server_negotiation.cc:407] Connection header received
0812 16:36:46.292220 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.292221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.292233 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.292262 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.292638 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43816: BlockingRecv error: recv got EOF from 127.0.0.1:43816 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 16:36:46.293323 13409 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:46.290742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47292 (local address 127.19.238.126:38159)
0812 16:36:46.290782 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.290783 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.292165 (+ 1382us) server_negotiation.cc:407] Connection header received
0812 16:36:46.292917 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.292918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.292929 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.292959 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.293268 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47292: BlockingRecv error: recv got EOF from 127.0.0.1:47292 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 16:36:46.295871 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:46.295002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48900 (local address 127.19.238.125:37309)
0812 16:36:46.295038 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.295039 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.295269 (+ 230us) server_negotiation.cc:407] Connection header received
0812 16:36:46.295395 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.295396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.295417 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.295448 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.295804 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48900: BlockingRecv error: recv got EOF from 127.0.0.1:48900 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 16:36:46.298890 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:46.298092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48902 (local address 127.19.238.125:37309)
0812 16:36:46.298126 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.298127 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.298408 (+ 281us) server_negotiation.cc:407] Connection header received
0812 16:36:46.298504 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.298504 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.298515 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.298544 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.298842 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48902: BlockingRecv error: recv got EOF from 127.0.0.1:48902 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 16:36:46.302747 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:46.301963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48904 (local address 127.19.238.125:37309)
0812 16:36:46.301997 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.301998 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.302240 (+ 242us) server_negotiation.cc:407] Connection header received
0812 16:36:46.302347 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.302348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.302359 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.302389 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.302700 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48904: BlockingRecv error: recv got EOF from 127.0.0.1:48904 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 16:36:46.308705 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:46.307895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48910 (local address 127.19.238.125:37309)
0812 16:36:46.307934 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.307936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.308185 (+ 249us) server_negotiation.cc:407] Connection header received
0812 16:36:46.308279 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.308280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.308291 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.308320 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.308655 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48910: BlockingRecv error: recv got EOF from 127.0.0.1:48910 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 16:36:46.318969 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:46.317986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48924 (local address 127.19.238.125:37309)
0812 16:36:46.318032 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.318034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.318364 (+ 330us) server_negotiation.cc:407] Connection header received
0812 16:36:46.318467 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.318468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.318480 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.318510 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.318910 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48924: BlockingRecv error: recv got EOF from 127.0.0.1:48924 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 16:36:46.337550 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:46.336422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48926 (local address 127.19.238.125:37309)
0812 16:36:46.336470 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.336472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.336798 (+ 326us) server_negotiation.cc:407] Connection header received
0812 16:36:46.336908 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.336909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.336923 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.336955 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.337489 (+ 534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48926: BlockingRecv error: recv got EOF from 127.0.0.1:48926 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 16:36:46.372238 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:46.371079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48928 (local address 127.19.238.125:37309)
0812 16:36:46.371128 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.371131 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.371490 (+ 359us) server_negotiation.cc:407] Connection header received
0812 16:36:46.371610 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.371612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.371625 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.371656 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.372177 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48928: BlockingRecv error: recv got EOF from 127.0.0.1:48928 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 16:36:46.438627 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:46.437783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48944 (local address 127.19.238.125:37309)
0812 16:36:46.437859 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.437861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.437943 (+ 82us) server_negotiation.cc:407] Connection header received
0812 16:36:46.438014 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.438015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.438027 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.438084 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.438546 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48944: BlockingRecv error: recv got EOF from 127.0.0.1:48944 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 16:36:46.568720 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:46.568010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48946 (local address 127.19.238.125:37309)
0812 16:36:46.568093 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.568095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.568106 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:36:46.568147 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.568147 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.568158 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.568215 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.568627 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48946: BlockingRecv error: recv got EOF from 127.0.0.1:48946 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 16:36:46.827035 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:46.826201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48954 (local address 127.19.238.125:37309)
0812 16:36:46.826279 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:46.826287 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:46.826302 (+ 15us) server_negotiation.cc:407] Connection header received
0812 16:36:46.826487 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:46.826489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:46.826501 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:46.826563 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:46.826969 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48954: BlockingRecv error: recv got EOF from 127.0.0.1:48954 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 16:36:47.200403 13617 catalog_manager.cc:1259] Loaded cluster ID: 79e43f431f4046f59818fed225ec57e7
I20260812 16:36:47.200493 13617 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1: loading cluster ID for follower catalog manager: success
I20260812 16:36:47.201460 13617 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1: acquiring CA information for follower catalog manager: success
I20260812 16:36:47.201777 13617 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c5ee9273031b49abbc8ca70234f51ce1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:36:47.247802 13652 catalog_manager.cc:1259] Loaded cluster ID: 79e43f431f4046f59818fed225ec57e7
I20260812 16:36:47.247963 13652 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab: loading cluster ID for follower catalog manager: success
I20260812 16:36:47.248764 13652 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab: acquiring CA information for follower catalog manager: success
I20260812 16:36:47.249092 13652 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 019112ee194148dca14b1596c3269eab: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:36:47.343286 13735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:47.340355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48968 (local address 127.19.238.125:37309)
0812 16:36:47.340495 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:47.340498 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:47.340509 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:36:47.340566 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:47.340567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:47.340621 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:47.340697 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:36:47.340838 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:47.341390 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:47.342275 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:47.342534 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:47.342592 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:47.342782 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:36:47.342913 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:36:47.343064 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:47.343094 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 16:36:47.343116 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260812 16:36:47.347832 13741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:47.345387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43822 (local address 127.19.238.124:34845)
0812 16:36:47.345491 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:47.345494 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:47.345511 (+ 17us) server_negotiation.cc:407] Connection header received
0812 16:36:47.345551 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:47.345551 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:47.345587 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:47.345660 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:36:47.345743 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:47.346276 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:47.347000 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:47.347244 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:47.347308 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:47.347449 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:36:47.347559 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:36:47.347624 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:47.347681 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 16:36:47.347699 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":54,"threads_started":1}
I20260812 16:36:47.352460 13735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:47.349594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48976 (local address 127.19.238.125:37309)
0812 16:36:47.349705 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:47.349708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:47.349797 (+ 89us) server_negotiation.cc:407] Connection header received
0812 16:36:47.349944 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:47.349945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:47.349959 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:47.349992 (+ 33us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:36:47.350200 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:47.350704 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:47.351645 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:47.352012 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:47.352098 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:47.352204 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:36:47.352239 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:36:47.352293 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:47.352357 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 16:36:47.352379 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 16:36:47.352861 13741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:47.349836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43832 (local address 127.19.238.124:34845)
0812 16:36:47.349878 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:47.349880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:47.350005 (+ 125us) server_negotiation.cc:407] Connection header received
0812 16:36:47.350087 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:47.350089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:47.350102 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:47.350168 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:36:47.350270 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:47.351063 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:47.352032 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:47.352354 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:47.352410 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:47.352609 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:36:47.352640 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:36:47.352665 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:47.352765 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 16:36:47.352791 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 16:36:47.353261 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:47.349593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47304 (local address 127.19.238.126:38159)
0812 16:36:47.349763 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:47.349765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:47.349774 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:36:47.349824 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:47.349825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:47.349880 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:47.349984 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:36:47.350221 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:47.351079 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:47.352179 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:47.352545 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:47.352621 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:47.352883 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:36:47.353011 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:36:47.353094 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:47.353148 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 16:36:47.353177 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":55,"threads_started":1}
I20260812 16:36:47.355177 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 13325
I20260812 16:36:47.362771 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 13415
I20260812 16:36:47.370057 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 13497
2026-08-12T16:36:47Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2921 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:36:47 dist-test-slave-mvvj krb5kdc[13753](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:36:47 dist-test-slave-mvvj krb5kdc[13753](info): set up 2 sockets
Aug 12 16:36:47 dist-test-slave-mvvj krb5kdc[13753](info): commencing operation
krb5kdc: starting...
W20260812 16:36:48.516789 20409 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/test-user.keytab.
Aug 12 16:36:48 dist-test-slave-mvvj krb5kdc[13753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:36:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:36:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.238.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
Entry for principal kudu/127.19.238.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab.
I20260812 16:36:48.643682 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:46191
--webserver_interface=127.19.238.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:35397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:46191,127.19.238.125:36973,127.19.238.124:35359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:36:48.726580 13769 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 16:36:48.726796 13769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:48.726814 13769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:48.726835 13769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:48.728019 13769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:48.728063 13769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:48.728078 13769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:48.728091 13769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:48.728103 13769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:48.729595 13769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:35397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:46191,127.19.238.125:36973,127.19.238.124:35359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.126.keytab
--principal=kudu/127.19.238.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.126:46191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.238.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:48.729795 13769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:48.729966 13769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:48.734862 13779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:48.734884 13776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:48.734884 13777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:48.735312 13769 server_base.cc:1034] running on GCE node
I20260812 16:36:48.735566 13769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:48.735911 13769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:48.737069 13769 hybrid_clock.cc:648] HybridClock initialized: now 1786552608737053 us; error 41 us; skew 500 ppm
Aug 12 16:36:48 dist-test-slave-mvvj krb5kdc[13753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552608, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:48.739693 13769 init.cc:377] Logged in from keytab as kudu/127.19.238.126@KRBTEST.COM (short username kudu)
I20260812 16:36:48.740222 13769 webserver.cc:460] Webserver started at http://127.19.238.126:36043/ using document root <none> and password file <none>
I20260812 16:36:48.740483 13769 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:48.740554 13769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:48.740706 13769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:48.741807 13769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-0/data/instance:
uuid: "09cfa07b862a487b9350f15f65b8d8ee"
format_stamp: "Formatted at 2026-08-12 16:36:48 on dist-test-slave-mvvj"
server_key: "2e1699dcd8fd61875348147ddb7df1ff"
server_key_iv: "14aabdb01fcb6c0d2d4ade500bfb0a7b"
server_key_version: "encryptionkey@0"
I20260812 16:36:48.742172 13769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-0/wal/instance:
uuid: "09cfa07b862a487b9350f15f65b8d8ee"
format_stamp: "Formatted at 2026-08-12 16:36:48 on dist-test-slave-mvvj"
server_key: "2e1699dcd8fd61875348147ddb7df1ff"
server_key_iv: "14aabdb01fcb6c0d2d4ade500bfb0a7b"
server_key_version: "encryptionkey@0"
I20260812 16:36:48.743656 13769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:36:48.744524 13786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:48.744771 13769 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:36:48.744899 13769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-0/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-0/wal
uuid: "09cfa07b862a487b9350f15f65b8d8ee"
format_stamp: "Formatted at 2026-08-12 16:36:48 on dist-test-slave-mvvj"
server_key: "2e1699dcd8fd61875348147ddb7df1ff"
server_key_iv: "14aabdb01fcb6c0d2d4ade500bfb0a7b"
server_key_version: "encryptionkey@0"
I20260812 16:36:48.744997 13769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:48.777351 13769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:48.779506 13769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:48.779697 13769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:48.784176 13769 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.126:46191
I20260812 16:36:48.784215 13846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.126:46191 every 8 connection(s)
I20260812 16:36:48.784657 13769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-0/data/info.pb
I20260812 16:36:48.785600 13848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:48.787222 13848 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:36:48.789516 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 13769
I20260812 16:36:48.789592 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-0/wal/instance
I20260812 16:36:48.789809 20409 external_mini_cluster.cc:1428] Setting key 043cb3f6f2d74bad79623e57f157dbd5
Aug 12 16:36:48 dist-test-slave-mvvj krb5kdc[13753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552608, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.238.125@KRBTEST.COM; defaulting to no policy
I20260812 16:36:48.798007 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:48.788047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46191 (local address 127.0.0.1:43164)
0812 16:36:48.788235 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:48.788271 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:48.788761 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:48.789122 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:48.789124 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:48.789391 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:48.789591 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:48.789597 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:48.790691 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:48.790692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:48.791349 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:48.791351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:48.791882 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:48.791884 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:48.791949 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:48.792587 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:48.792596 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:48.794475 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:48.796441 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:48.796449 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:48.796451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:48.796654 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:48.796923 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:48.796924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:48.796925 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:48.797016 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:48.797386 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:48.797392 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:48.797454 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 16:36:48.797565 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 16:36:48.797804 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":96,"thread_start_us":102,"threads_started":1}
I20260812 16:36:48.798009 13851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:48.788047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43164 (local address 127.19.238.126:46191)
0812 16:36:48.788905 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:48.788909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:48.788919 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:36:48.788982 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:48.788983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:48.788998 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:48.789063 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:48.789706 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:48.790588 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:48.791442 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:48.791762 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:48.791934 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:48.794591 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:48.794600 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:48.794601 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:48.794614 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:48.796337 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:48.796754 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:48.796757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:48.796757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:48.796851 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:48.797091 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:48.797092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:48.797092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:48.797308 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:48.797407 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:48.797601 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 16:36:48.797738 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":826,"spinlock_wait_cycles":8448,"thread_start_us":104,"threads_started":1}
I20260812 16:36:48.798769 13848 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:36:48.799170 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:48.799044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:36973 (local address 127.0.0.1:37912)
0812 16:36:48.799098 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:48.799150 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.125:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 16:36:48.799342 13788 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.238.125:36973: connect: Connection refused (error 111)
I20260812 16:36:48.799801 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:48.799715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:36973 (local address 127.0.0.1:37924)
0812 16:36:48.799760 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:48.799784 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.125:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 16:36:48.799940 13848 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.125:36973: Network error: Client connection negotiation failed: client connection to 127.19.238.125:36973: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.125.keytab.
Entry for principal kudu/127.19.238.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.125.keytab.
I20260812 16:36:48.819167 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.125:36973
--webserver_interface=127.19.238.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.125.keytab
--principal=kudu/127.19.238.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:35397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:46191,127.19.238.125:36973,127.19.238.124:35359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:36:48.849195 13848 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } attempt: 1
I20260812 16:36:48.849799 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:48.849657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:36973 (local address 127.0.0.1:37934)
0812 16:36:48.849724 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:48.849767 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.125:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 16:36:48.850184 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:48.850082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:36973 (local address 127.0.0.1:37936)
0812 16:36:48.850130 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:48.850163 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.125:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 16:36:48.850284 13848 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.125:36973: Network error: Client connection negotiation failed: client connection to 127.19.238.125:36973: connect: Connection refused (error 111)
W20260812 16:36:48.902868 13855 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 16:36:48.903126 13855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:48.903157 13855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:48.903200 13855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:48.904376 13855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:48.904457 13855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:48.904485 13855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:48.904518 13855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:48.904539 13855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:48.906037 13855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:35397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:46191,127.19.238.125:36973,127.19.238.124:35359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.125.keytab
--principal=kudu/127.19.238.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.125:36973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.238.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:48.906324 13855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:48.906589 13855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:48.912002 13864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:48.912014 13861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:48.912022 13855 server_base.cc:1034] running on GCE node
W20260812 16:36:48.912034 13862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:48.912513 13855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:48.912814 13855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:48.913995 13855 hybrid_clock.cc:648] HybridClock initialized: now 1786552608913915 us; error 103 us; skew 500 ppm
Aug 12 16:36:48 dist-test-slave-mvvj krb5kdc[13753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552608, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:48.916575 13855 init.cc:377] Logged in from keytab as kudu/127.19.238.125@KRBTEST.COM (short username kudu)
I20260812 16:36:48.917083 13855 webserver.cc:460] Webserver started at http://127.19.238.125:46621/ using document root <none> and password file <none>
I20260812 16:36:48.917317 13855 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:48.917385 13855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:48.917547 13855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:48.918443 13848 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } attempt: 2
I20260812 16:36:48.918745 13855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-1/data/instance:
uuid: "5d64fae61919498898d0fffb5e835cbb"
format_stamp: "Formatted at 2026-08-12 16:36:48 on dist-test-slave-mvvj"
server_key: "a810ee97cb26ae5b00cfe3afa756649d"
server_key_iv: "5f194c61ae956d4c39b717072f618cac"
server_key_version: "encryptionkey@0"
I20260812 16:36:48.918865 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:48.918731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:36973 (local address 127.0.0.1:37950)
0812 16:36:48.918796 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:48.918836 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.125:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 16:36:48.919137 13855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-1/wal/instance:
uuid: "5d64fae61919498898d0fffb5e835cbb"
format_stamp: "Formatted at 2026-08-12 16:36:48 on dist-test-slave-mvvj"
server_key: "a810ee97cb26ae5b00cfe3afa756649d"
server_key_iv: "5f194c61ae956d4c39b717072f618cac"
server_key_version: "encryptionkey@0"
I20260812 16:36:48.919242 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:48.919131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:36973 (local address 127.0.0.1:37966)
0812 16:36:48.919180 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:48.919206 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.125:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 16:36:48.919322 13848 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.125:36973: Network error: Client connection negotiation failed: client connection to 127.19.238.125:36973: connect: Connection refused (error 111)
I20260812 16:36:48.920776 13855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 16:36:48.921577 13872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:48.921764 13855 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 16:36:48.921872 13855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-1/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-1/wal
uuid: "5d64fae61919498898d0fffb5e835cbb"
format_stamp: "Formatted at 2026-08-12 16:36:48 on dist-test-slave-mvvj"
server_key: "a810ee97cb26ae5b00cfe3afa756649d"
server_key_iv: "5f194c61ae956d4c39b717072f618cac"
server_key_version: "encryptionkey@0"
I20260812 16:36:48.921964 13855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:48.941381 13855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:48.943491 13855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:48.943670 13855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:48.948592 13855 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.125:36973
I20260812 16:36:48.948686 13934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.125:36973 every 8 connection(s)
I20260812 16:36:48.949043 13855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-1/data/info.pb
I20260812 16:36:48.950037 13935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:48.951684 13935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:36:48.954963 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 13855
I20260812 16:36:48.955072 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-1/wal/instance
I20260812 16:36:48.955255 20409 external_mini_cluster.cc:1428] Setting key 823ac4bde10c84712ae5c9858d7c4eb7
Aug 12 16:36:48 dist-test-slave-mvvj krb5kdc[13753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552608, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.125@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:36:48.962569 13851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:48.952526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43178 (local address 127.19.238.126:46191)
0812 16:36:48.952603 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:48.952606 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:48.952804 (+ 198us) server_negotiation.cc:407] Connection header received
0812 16:36:48.953337 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:48.953338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:48.953354 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:48.953418 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:48.954075 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:48.954590 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:48.956125 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:48.956401 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:48.956477 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:48.959055 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:48.959057 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:48.959057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:48.959066 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:48.960940 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:48.961448 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:48.961450 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:48.961450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:48.961525 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:48.961778 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:48.961779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:48.961780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:48.961937 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:48.962026 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:48.962339 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 16:36:48.962438 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 16:36:48.962724 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:48.952488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46191 (local address 127.0.0.1:43178)
0812 16:36:48.952679 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:48.952717 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:48.953228 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:48.953493 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:48.953498 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:48.953769 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:48.953975 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:48.953980 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:48.954691 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:48.954693 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:48.956018 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:48.956023 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:48.956500 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:48.956501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:48.956650 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:48.957292 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:48.957301 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:48.958943 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:48.961053 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:48.961061 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:48.961062 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:48.961361 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:48.961600 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:48.961601 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:48.961602 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:48.961710 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:48.962052 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:48.962059 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:48.962191 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 16:36:48.962286 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 16:36:48.962499 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
I20260812 16:36:48.963348 13935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.238.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.238.124@KRBTEST.COM" created.
Aug 12 16:36:48 dist-test-slave-mvvj krb5kdc[13753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552608, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.125@KRBTEST.COM for kudu/127.19.238.125@KRBTEST.COM
I20260812 16:36:48.971648 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:48.963747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37978 (local address 127.19.238.125:36973)
0812 16:36:48.963992 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:48.963995 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:48.964013 (+ 18us) server_negotiation.cc:407] Connection header received
0812 16:36:48.964032 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:48.964033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:48.964044 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:48.964127 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:48.964435 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:48.965122 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:48.965843 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:48.966161 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:48.966254 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:48.968290 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:48.968298 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:48.968299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:48.968307 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:48.970324 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:48.970798 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:48.970806 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:48.970806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:48.970887 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:48.971119 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:48.971120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:48.971120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:48.971267 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:48.971334 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:48.971493 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 16:36:48.971566 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":168,"threads_started":1}
I20260812 16:36:48.971802 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:48.963642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:36973 (local address 127.0.0.1:37978)
0812 16:36:48.963691 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:48.963703 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:48.963744 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:48.964141 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:48.964142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:48.964282 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:48.964345 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:48.964347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:48.965216 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:48.965217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:48.965750 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:48.965752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:48.966255 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:48.966256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:48.966339 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:48.966714 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:48.966718 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:48.968191 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:48.970408 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:48.970410 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:48.970411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:48.970688 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:48.970978 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:48.970979 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:48.970979 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:48.971052 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:48.971335 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:48.971336 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:48.971389 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 16:36:48.971469 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 16:36:48.971635 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":56}
I20260812 16:36:48.972509 13935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:36:48.972869 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:48.972721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:35359 (local address 127.0.0.1:40650)
0812 16:36:48.972778 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:48.972835 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:35359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 16:36:48.973017 13875 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.238.124:35359: connect: Connection refused (error 111)
I20260812 16:36:48.973551 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:48.973429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:35359 (local address 127.0.0.1:40656)
0812 16:36:48.973486 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:48.973520 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:35359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 16:36:48.973687 13935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.124:35359: Network error: Client connection negotiation failed: client connection to 127.19.238.124:35359: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.238.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.124.keytab.
Entry for principal kudu/127.19.238.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.124.keytab.
I20260812 16:36:48.987674 20409 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt79009/build/release/bin/kudu
/tmp/dist-test-taskt79009/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.124:35359
--webserver_interface=127.19.238.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.124.keytab
--principal=kudu/127.19.238.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.238.84:35397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.238.126:46191,127.19.238.125:36973,127.19.238.124:35359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:36:49.010481 13848 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } attempt: 3
Aug 12 16:36:49 dist-test-slave-mvvj krb5kdc[13753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552608, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for kudu/127.19.238.125@KRBTEST.COM
I20260812 16:36:49.018373 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.010886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:36973 (local address 127.0.0.1:37990)
0812 16:36:49.010966 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:49.010981 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:49.011036 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:49.011229 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:49.011229 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:49.011395 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:49.011467 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:49.011468 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.012468 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.012469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:49.012982 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:49.012983 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.013468 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.013470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:49.013524 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:49.014026 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:49.014031 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:49.015507 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:49.017353 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:49.017354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:49.017355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:49.017552 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:49.017765 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:49.017765 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:49.017766 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:49.017818 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:49.018118 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:49.018118 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:49.018156 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 16:36:49.018204 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 16:36:49.018285 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 16:36:49.018468 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.010999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37990 (local address 127.19.238.125:36973)
0812 16:36:49.011068 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.011070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.011081 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:36:49.011116 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.011117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.011133 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.011214 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.011549 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.012367 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.013061 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.013391 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.013464 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:49.015608 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:49.015609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:49.015609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:49.015618 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:49.017259 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.017628 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.017629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.017630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.017696 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.017877 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.017877 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.017878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.018042 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:49.018116 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:49.018254 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 16:36:49.018367 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 16:36:49.018967 13848 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:36:49.019330 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.019233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:35359 (local address 127.0.0.1:40668)
0812 16:36:49.019279 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:49.019309 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:35359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 16:36:49.019685 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.019594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:35359 (local address 127.0.0.1:40684)
0812 16:36:49.019639 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:49.019669 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:35359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 16:36:49.019771 13848 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.124:35359: Network error: Client connection negotiation failed: client connection to 127.19.238.124:35359: connect: Connection refused (error 111)
I20260812 16:36:49.022929 13935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } attempt: 1
I20260812 16:36:49.023344 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.023225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:35359 (local address 127.0.0.1:40700)
0812 16:36:49.023280 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:49.023316 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:35359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 16:36:49.023696 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.023602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:35359 (local address 127.0.0.1:40706)
0812 16:36:49.023647 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:49.023673 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:35359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 16:36:49.023810 13935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.124:35359: Network error: Client connection negotiation failed: client connection to 127.19.238.124:35359: connect: Connection refused (error 111)
I20260812 16:36:49.050921 13848 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } attempt: 1
I20260812 16:36:49.051484 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.051290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:35359 (local address 127.0.0.1:40716)
0812 16:36:49.051413 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:49.051451 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:35359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260812 16:36:49.051928 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.051784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:35359 (local address 127.0.0.1:40720)
0812 16:36:49.051870 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:49.051899 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:35359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260812 16:36:49.052024 13848 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.124:35359: Network error: Client connection negotiation failed: client connection to 127.19.238.124:35359: connect: Connection refused (error 111)
W20260812 16:36:49.075094 13944 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 16:36:49.075363 13944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:36:49.075402 13944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:36:49.075439 13944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:36:49.076712 13944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:36:49.076778 13944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:36:49.076805 13944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:36:49.076835 13944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:36:49.076858 13944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:36:49.078373 13944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.238.84:35397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.238.126:46191,127.19.238.125:36973,127.19.238.124:35359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/krb5kdc/kudu_127.19.238.124.keytab
--principal=kudu/127.19.238.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.238.124:35359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.238.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 15:46:00 UTC on ccycloud.quasar-illntx.root.comops.site
build id 4583010
I20260812 16:36:49.078660 13944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:36:49.078890 13944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:36:49.084184 13955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:49.084233 13944 server_base.cc:1034] running on GCE node
W20260812 16:36:49.084168 13953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:36:49.084164 13951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:36:49.084827 13944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:36:49.085101 13944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:36:49.086247 13944 hybrid_clock.cc:648] HybridClock initialized: now 1786552609086222 us; error 37 us; skew 500 ppm
Aug 12 16:36:49 dist-test-slave-mvvj krb5kdc[13753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552609, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:36:49.088879 13944 init.cc:377] Logged in from keytab as kudu/127.19.238.124@KRBTEST.COM (short username kudu)
I20260812 16:36:49.089381 13944 webserver.cc:460] Webserver started at http://127.19.238.124:36559/ using document root <none> and password file <none>
I20260812 16:36:49.089617 13944 fs_manager.cc:359] Metadata directory not provided
I20260812 16:36:49.089665 13944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:36:49.089818 13944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:36:49.090952 13944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-2/data/instance:
uuid: "638775196708415f931cd7d425669495"
format_stamp: "Formatted at 2026-08-12 16:36:49 on dist-test-slave-mvvj"
server_key: "e190cdc13563db61ce6884be410f37c3"
server_key_iv: "e232ed2f0e12aa7169788755ba51e729"
server_key_version: "encryptionkey@0"
I20260812 16:36:49.091339 13944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-2/wal/instance:
uuid: "638775196708415f931cd7d425669495"
format_stamp: "Formatted at 2026-08-12 16:36:49 on dist-test-slave-mvvj"
server_key: "e190cdc13563db61ce6884be410f37c3"
server_key_iv: "e232ed2f0e12aa7169788755ba51e729"
server_key_version: "encryptionkey@0"
I20260812 16:36:49.091961 13935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } attempt: 2
I20260812 16:36:49.092378 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.092249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:35359 (local address 127.0.0.1:40734)
0812 16:36:49.092313 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:49.092351 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:35359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 16:36:49.092805 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.092695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:35359 (local address 127.0.0.1:40740)
0812 16:36:49.092745 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:49.092779 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.238.124:35359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 16:36:49.092933 13935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.238.124:35359: Network error: Client connection negotiation failed: client connection to 127.19.238.124:35359: connect: Connection refused (error 111)
I20260812 16:36:49.093072 13944 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 16:36:49.094027 13965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:36:49.094259 13944 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 16:36:49.094355 13944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-2/data,/tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-2/wal
uuid: "638775196708415f931cd7d425669495"
format_stamp: "Formatted at 2026-08-12 16:36:49 on dist-test-slave-mvvj"
server_key: "e190cdc13563db61ce6884be410f37c3"
server_key_iv: "e232ed2f0e12aa7169788755ba51e729"
server_key_version: "encryptionkey@0"
I20260812 16:36:49.094431 13944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:36:49.113159 13944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:36:49.115096 13944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:36:49.115247 13944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:36:49.119473 13944 rpc_server.cc:307] RPC server started. Bound to: 127.19.238.124:35359
I20260812 16:36:49.119551 14019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.238.124:35359 every 8 connection(s)
I20260812 16:36:49.119993 13944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-2/data/info.pb
I20260812 16:36:49.120859 14020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:36:49.122403 14020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:36:49.123629 20409 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt79009/build/release/bin/kudu as pid 13944
I20260812 16:36:49.123759 20409 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt79009/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786552327177731-20409-0/minicluster-data/master-2/wal/instance
I20260812 16:36:49.124032 20409 external_mini_cluster.cc:1428] Setting key cbbae7eb1f49f14be442ae946b251de9
I20260812 16:36:49.127187 13848 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } attempt: 2
Aug 12 16:36:49 dist-test-slave-mvvj krb5kdc[13753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552609, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.124@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
Aug 12 16:36:49 dist-test-slave-mvvj krb5kdc[13753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.124@KRBTEST.COM
Aug 12 16:36:49 dist-test-slave-mvvj krb5kdc[13753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552608, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.126@KRBTEST.COM for kudu/127.19.238.124@KRBTEST.COM
I20260812 16:36:49.134558 13851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.123118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43194 (local address 127.19.238.126:46191)
0812 16:36:49.123202 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.123204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.123424 (+ 220us) server_negotiation.cc:407] Connection header received
0812 16:36:49.124260 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.124261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.124272 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.124317 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.125179 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.125956 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.127251 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.127531 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.127602 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:49.130401 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:49.130402 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:49.130403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:49.130409 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:49.132362 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.133385 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.133386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.133387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.133446 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.133785 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.133786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.133786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.133949 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:49.134059 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:49.134358 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 16:36:49.134443 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 16:36:49.135006 14022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.123104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.126:46191 (local address 127.0.0.1:43194)
0812 16:36:49.123283 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:49.123334 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:49.124096 (+ 762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:49.124405 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:49.124409 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:49.124798 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:49.125108 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:49.125112 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.126063 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.126064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:49.127140 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:49.127142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.127678 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.127679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:49.127862 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:49.128489 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:49.128500 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:49.130296 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:49.132726 (+ 2430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:49.132735 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:49.132738 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:49.133277 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:49.133562 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:49.133563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:49.133564 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:49.133700 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:49.134053 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:49.134060 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:49.134213 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 16:36:49.134323 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 16:36:49.134551 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
I20260812 16:36:49.135638 14020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:36:49.136278 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.127520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:35359 (local address 127.0.0.1:40764)
0812 16:36:49.127560 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:49.127570 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:49.127612 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:49.127881 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:49.127882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:49.128082 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:49.128121 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:49.128123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.129041 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.129042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:49.129846 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:49.129849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.130328 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.130329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:49.130399 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:49.130801 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:49.130807 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:49.132365 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:49.134973 (+ 2608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:49.134975 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:49.134975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:49.135222 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:49.135482 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:49.135483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:49.135483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:49.135554 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:49.135936 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:49.135937 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:49.135985 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 16:36:49.136068 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 16:36:49.136175 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 16:36:49.136639 14024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.124652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40756 (local address 127.19.238.124:35359)
0812 16:36:49.124847 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.124851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.124872 (+ 21us) server_negotiation.cc:407] Connection header received
0812 16:36:49.124893 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.124895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.124914 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.125001 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.125846 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.127115 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.128390 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.128721 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.128813 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:49.132306 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:49.132315 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:49.132315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:49.132322 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:49.134937 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.135314 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.135317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.135318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.135421 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.135643 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.135643 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.135644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.135785 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:49.136332 (+ 547us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:49.136381 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 16:36:49.136538 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":165,"thread_start_us":95,"threads_started":1}
I20260812 16:36:49.136803 14025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.127566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40764 (local address 127.19.238.124:35359)
0812 16:36:49.127731 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.127733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.127739 (+ 6us) server_negotiation.cc:407] Connection header received
0812 16:36:49.127759 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.127760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.127769 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.127864 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.128219 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.128946 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.129942 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.130250 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.130324 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:49.132477 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:49.132479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:49.132479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:49.132486 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:49.134882 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.135314 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.135316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.135317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.135396 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.135609 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.135610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.135611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.135876 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:49.135969 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:49.136410 (+ 441us) server_negotiation.cc:299] Negotiation successful
0812 16:36:49.136662 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":123,"server-negotiator.queue_time_us":136,"thread_start_us":108,"threads_started":1}
I20260812 16:36:49.138870 13848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee: Bootstrap starting.
I20260812 16:36:49.139694 13848 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:49.140043 13848 log.cc:824] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee: Log is configured to *not* fsync() on all Append() calls
Aug 12 16:36:49 dist-test-slave-mvvj krb5kdc[13753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552609, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.124@KRBTEST.COM for kudu/127.19.238.125@KRBTEST.COM
I20260812 16:36:49.141016 13848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee: No bootstrap required, opened a new log
Aug 12 16:36:49 dist-test-slave-mvvj krb5kdc[13753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.126@KRBTEST.COM
I20260812 16:36:49.143373 13848 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cfa07b862a487b9350f15f65b8d8ee" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } } peers { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } } peers { permanent_uuid: "638775196708415f931cd7d425669495" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } }
I20260812 16:36:49.143594 13848 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09cfa07b862a487b9350f15f65b8d8ee, State: Initialized, Role: FOLLOWER
I20260812 16:36:49.143697 13848 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cfa07b862a487b9350f15f65b8d8ee" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } } peers { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } } peers { permanent_uuid: "638775196708415f931cd7d425669495" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } }
I20260812 16:36:49.144106 13848 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:49.144394 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.136057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38000 (local address 127.19.238.125:36973)
0812 16:36:49.136111 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.136113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.136122 (+ 9us) server_negotiation.cc:407] Connection header received
0812 16:36:49.136314 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.136316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.136329 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.136383 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.136834 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.137508 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.138723 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.139018 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.139093 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:49.141227 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:49.141229 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:49.141229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:49.141237 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:49.142971 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.143389 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.143390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.143391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.143454 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.143708 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.143709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.143709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.143896 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:49.143948 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:49.144193 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 16:36:49.144278 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 16:36:49.144747 14022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.135944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.125:36973 (local address 127.0.0.1:38000)
0812 16:36:49.136044 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:49.136054 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:49.136147 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:49.136438 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:49.136439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:49.136688 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:49.136728 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:49.136730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.137687 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.137688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:49.138623 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:49.138626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.139107 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.139108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:49.139171 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:49.139601 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:49.139606 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:49.141127 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:49.143082 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:49.143084 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:49.143084 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:49.143289 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:49.143559 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:49.143560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:49.143561 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:49.143629 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:49.144019 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:49.144020 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:49.144068 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 16:36:49.144527 (+ 459us) client_negotiation.cc:240] Negotiation successful
0812 16:36:49.144640 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 16:36:49.145129 14020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:36:49.144049 14027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cfa07b862a487b9350f15f65b8d8ee" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } } peers { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } } peers { permanent_uuid: "638775196708415f931cd7d425669495" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } } }
I20260812 16:36:49.145437 14027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:36:49.146309 13851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.137418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43206 (local address 127.19.238.126:46191)
0812 16:36:49.137457 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.137458 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.137466 (+ 8us) server_negotiation.cc:407] Connection header received
0812 16:36:49.137484 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.137485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.137493 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.137546 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.138184 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.138857 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.139759 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.140083 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.140132 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:49.142687 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:49.142688 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:49.142689 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:49.142697 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:49.144530 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.145089 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.145090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.145091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.145153 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.145553 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.145554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.145554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.145688 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:49.145794 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:49.146125 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 16:36:49.146210 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 16:36:49.147011 14040 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:36:49.147064 14040 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 12 16:36:49 dist-test-slave-mvvj krb5kdc[13753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552609, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.124@KRBTEST.COM for kudu/127.19.238.124@KRBTEST.COM
I20260812 16:36:49.147039 13769 master_runner.cc:186] Error getting master registration for 127.19.238.125:36973: OK, instance_id { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" instance_seqno: 1786552608947034 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 16:36:49.150194 13769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 12 16:36:49 dist-test-slave-mvvj krb5kdc[13753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.238.125@KRBTEST.COM
I20260812 16:36:49.152961 14022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.145433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:35359 (local address 127.0.0.1:40772)
0812 16:36:49.145497 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:49.145507 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:49.145548 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:49.145772 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:49.145773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:49.145936 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:49.145975 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:49.145978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.146874 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.146875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:49.147605 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:49.147608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.148114 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.148115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:49.148174 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:49.148655 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:49.148660 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:49.149888 (+ 1228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:49.151656 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:49.151657 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:49.151657 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:49.151901 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:49.152099 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:49.152100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:49.152100 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:49.152145 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:49.152581 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:49.152581 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:49.152622 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 16:36:49.152711 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 16:36:49.152821 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 16:36:49.153258 14024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.145455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40772 (local address 127.19.238.124:35359)
0812 16:36:49.145651 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.145653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.145663 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:36:49.145685 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.145686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.145697 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.145754 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.146078 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.146793 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.147698 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.148034 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.148112 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:49.149983 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:49.149984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:49.149985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:49.149991 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:49.151565 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.151984 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.151985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.151985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.152034 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.152265 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.152266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.152267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.152470 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:49.152533 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:49.152765 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 16:36:49.152872 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":174}
I20260812 16:36:49.154929 14020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495: Bootstrap starting.
I20260812 16:36:49.155777 14020 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:49.155802 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.147054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38014 (local address 127.19.238.125:36973)
0812 16:36:49.147089 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.147091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.147193 (+ 102us) server_negotiation.cc:407] Connection header received
0812 16:36:49.147253 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.147254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.147263 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.147311 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.147856 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.148599 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.149524 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.149807 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.149886 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:49.152682 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:49.152684 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:49.152684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:49.152692 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:49.154401 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.154833 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.154834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.154834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.154897 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.155192 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.155193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.155194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.155334 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:49.155532 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:49.155625 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 16:36:49.155706 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 16:36:49.156178 14020 log.cc:824] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:49.157159 14020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495: No bootstrap required, opened a new log
I20260812 16:36:49.157310 20409 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:36:49.158326 13819 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "09cfa07b862a487b9350f15f65b8d8ee"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43206
I20260812 16:36:49.158499 13819 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 16:36:49.158557 13819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:49.159147 14020 raft_consensus.cc:348] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cfa07b862a487b9350f15f65b8d8ee" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } } peers { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } } peers { permanent_uuid: "638775196708415f931cd7d425669495" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } }
I20260812 16:36:49.159303 14020 raft_consensus.cc:729] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 638775196708415f931cd7d425669495, State: Initialized, Role: FOLLOWER
I20260812 16:36:49.159399 14020 consensus_queue.cc:260] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cfa07b862a487b9350f15f65b8d8ee" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } } peers { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } } peers { permanent_uuid: "638775196708415f931cd7d425669495" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } }
I20260812 16:36:49.159543 13819 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cfa07b862a487b9350f15f65b8d8ee" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } } peers { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } } peers { permanent_uuid: "638775196708415f931cd7d425669495" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } }
I20260812 16:36:49.159799 14020 sys_catalog.cc:564] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:49.159857 13819 leader_election.cc:288] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [CANDIDATE]: Term 1 election: Requested vote from peers 5d64fae61919498898d0fffb5e835cbb (127.19.238.125:36973), 638775196708415f931cd7d425669495 (127.19.238.124:35359)
I20260812 16:36:49.159765 14044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cfa07b862a487b9350f15f65b8d8ee" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } } peers { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } } peers { permanent_uuid: "638775196708415f931cd7d425669495" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } } }
I20260812 16:36:49.160037 14044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:36:49.160260 13993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09cfa07b862a487b9350f15f65b8d8ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "638775196708415f931cd7d425669495"
I20260812 16:36:49.160379 13993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:49.161285 13993 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09cfa07b862a487b9350f15f65b8d8ee in term 1.
I20260812 16:36:49.161556 13789 leader_election.cc:302] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09cfa07b862a487b9350f15f65b8d8ee, 638775196708415f931cd7d425669495; no voters:
I20260812 16:36:49.161862 14027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:36:49.162045 14027 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [term 1 LEADER]: Becoming Leader. State: Replica: 09cfa07b862a487b9350f15f65b8d8ee, State: Running, Role: LEADER
I20260812 16:36:49.162142 14027 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cfa07b862a487b9350f15f65b8d8ee" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } } peers { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } } peers { permanent_uuid: "638775196708415f931cd7d425669495" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } }
W20260812 16:36:49.162588 14056 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260812 16:36:49.162595 14027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09cfa07b862a487b9350f15f65b8d8ee. Latest consensus state: current_term: 1 leader_uuid: "09cfa07b862a487b9350f15f65b8d8ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cfa07b862a487b9350f15f65b8d8ee" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } } peers { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } } peers { permanent_uuid: "638775196708415f931cd7d425669495" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } } }
W20260812 16:36:49.162667 14056 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:36:49.162709 14027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:49.162967 14065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:36:49.163193 14065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:36:49.164119 13944 master_runner.cc:186] Error getting master registration for 127.19.238.125:36973: OK, instance_id { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" instance_seqno: 1786552608947034 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 16:36:49.165755 13944 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 16:36:49.174660 13993 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495 [term 1 FOLLOWER]: Refusing update from remote peer 09cfa07b862a487b9350f15f65b8d8ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:36:49.174990 14027 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "638775196708415f931cd7d425669495" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:36:49.176369 14044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09cfa07b862a487b9350f15f65b8d8ee. Latest consensus state: current_term: 1 leader_uuid: "09cfa07b862a487b9350f15f65b8d8ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cfa07b862a487b9350f15f65b8d8ee" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } } peers { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } } peers { permanent_uuid: "638775196708415f931cd7d425669495" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } } }
I20260812 16:36:49.176502 14044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:36:49.178222 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.168205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38030 (local address 127.19.238.125:36973)
0812 16:36:49.168298 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.168300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.168557 (+ 257us) server_negotiation.cc:407] Connection header received
0812 16:36:49.168680 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.168681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.168695 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.168730 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.169609 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.170322 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.171336 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.171649 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.171697 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:49.174295 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:49.174297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:49.174298 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:49.174316 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:49.176083 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.176977 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.176978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.176979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.177055 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.177415 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.177416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.177417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.177577 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:49.177654 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:49.178037 (+ 383us) server_negotiation.cc:299] Negotiation successful
0812 16:36:49.178122 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 16:36:49.182650 13851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.168503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43216 (local address 127.19.238.126:46191)
0812 16:36:49.168560 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.168562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.168734 (+ 172us) server_negotiation.cc:407] Connection header received
0812 16:36:49.168881 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.168894 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.168904 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.168935 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.170025 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.170758 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.172239 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.172544 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.172617 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:49.178507 (+ 5890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:49.178508 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:49.178508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:49.178516 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:49.180569 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.181356 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.181358 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.181358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.181427 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.181824 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.181839 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.181840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.181983 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:49.182060 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:49.182453 (+ 393us) server_negotiation.cc:299] Negotiation successful
0812 16:36:49.182550 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 16:36:49.184073 14024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.168789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40784 (local address 127.19.238.124:35359)
0812 16:36:49.168838 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.168840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.172324 (+ 3484us) server_negotiation.cc:407] Connection header received
0812 16:36:49.172423 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.172424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.172435 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.172477 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.173403 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.174131 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.178806 (+ 4675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.179128 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.179171 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:49.181022 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:49.181023 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:49.181023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:49.181030 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:49.182724 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.183119 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.183120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.183120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.183181 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.183375 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.183376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.183376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.183555 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:49.183639 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:49.183885 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 16:36:49.183978 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 16:36:49.184090 13935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } attempt: 3
I20260812 16:36:49.185396 14057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "09cfa07b862a487b9350f15f65b8d8ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cfa07b862a487b9350f15f65b8d8ee" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } } peers { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } } peers { permanent_uuid: "638775196708415f931cd7d425669495" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } } }
I20260812 16:36:49.185534 14057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:49.185681 14066 mvcc.cc:205] Tried to move back new op lower bound from 7317719487174782976 to 7317719487128838144. Current Snapshot: MvccSnapshot[applied={T|T < 7317719487174782976}]
I20260812 16:36:49.186013 14044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "09cfa07b862a487b9350f15f65b8d8ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cfa07b862a487b9350f15f65b8d8ee" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } } peers { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } } peers { permanent_uuid: "638775196708415f931cd7d425669495" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } } }
I20260812 16:36:49.186105 14044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:36:49.186300 14070 mvcc.cc:205] Tried to move back new op lower bound from 7317719487174782976 to 7317719487128838144. Current Snapshot: MvccSnapshot[applied={T|T < 7317719487174782976}]
Aug 12 16:36:49 dist-test-slave-mvvj krb5kdc[13753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786552608, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.238.125@KRBTEST.COM for kudu/127.19.238.124@KRBTEST.COM
I20260812 16:36:49.193204 14024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.184748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40794 (local address 127.19.238.124:35359)
0812 16:36:49.184863 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.184865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.184872 (+ 7us) server_negotiation.cc:407] Connection header received
0812 16:36:49.184892 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.184893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.184904 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.184958 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.185844 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.186527 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.187518 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.187823 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.187932 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:49.190223 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:36:49.190224 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:36:49.190224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:36:49.190232 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:36:49.191974 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.192389 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.192390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.192391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.192465 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:36:49.192672 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:36:49.192672 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:36:49.192673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:36:49.192817 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:36:49.192925 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:49.193018 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 16:36:49.193102 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260812 16:36:49.193208 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:49.184337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.238.124:35359 (local address 127.0.0.1:40794)
0812 16:36:49.184618 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 16:36:49.184629 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 16:36:49.184675 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:36:49.184981 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:36:49.184983 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:36:49.185155 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:36:49.185209 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:49.185211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.186608 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.186609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:49.187419 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:36:49.187422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:36:49.187932 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:36:49.187933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:36:49.187992 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:49.188468 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:36:49.188473 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:36:49.190118 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:36:49.192063 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:49.192065 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:49.192065 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:49.192302 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:49.192539 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:36:49.192540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:36:49.192540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:36:49.192605 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:36:49.192879 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:36:49.192879 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:36:49.192928 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 16:36:49.192995 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 16:36:49.193108 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258}
I20260812 16:36:49.195400 13935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb: Bootstrap starting.
I20260812 16:36:49.196262 13935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb: Neither blocks nor log segments found. Creating new log.
I20260812 16:36:49.196677 14065 catalog_manager.cc:1347] Generated new cluster ID: 4a770b999f684c80ae49afb12e484e97
I20260812 16:36:49.196738 14065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:36:49.197537 13935 log.cc:824] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb: Log is configured to *not* fsync() on all Append() calls
I20260812 16:36:49.198612 13935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb: No bootstrap required, opened a new log
I20260812 16:36:49.200538 13935 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cfa07b862a487b9350f15f65b8d8ee" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } } peers { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } } peers { permanent_uuid: "638775196708415f931cd7d425669495" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } }
I20260812 16:36:49.200670 13935 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d64fae61919498898d0fffb5e835cbb, State: Initialized, Role: FOLLOWER
I20260812 16:36:49.200744 13935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cfa07b862a487b9350f15f65b8d8ee" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } } peers { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } } peers { permanent_uuid: "638775196708415f931cd7d425669495" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } }
I20260812 16:36:49.201054 13935 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:36:49.201090 13903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09cfa07b862a487b9350f15f65b8d8ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5d64fae61919498898d0fffb5e835cbb"
I20260812 16:36:49.201217 13903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:36:49.202087 13903 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09cfa07b862a487b9350f15f65b8d8ee in term 1.
I20260812 16:36:49.202294 13900 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb [term 1 FOLLOWER]: Refusing update from remote peer 09cfa07b862a487b9350f15f65b8d8ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:36:49.202637 14075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cfa07b862a487b9350f15f65b8d8ee" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } } peers { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } } peers { permanent_uuid: "638775196708415f931cd7d425669495" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } } }
I20260812 16:36:49.202822 14075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:36:49.203073 14027 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 16:36:49.205284 14090 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:36:49.205339 14090 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:36:49.205428 14093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "09cfa07b862a487b9350f15f65b8d8ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cfa07b862a487b9350f15f65b8d8ee" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } } peers { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } } peers { permanent_uuid: "638775196708415f931cd7d425669495" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } } }
I20260812 16:36:49.205497 14093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:36:49.206003 14075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09cfa07b862a487b9350f15f65b8d8ee. Latest consensus state: current_term: 1 leader_uuid: "09cfa07b862a487b9350f15f65b8d8ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cfa07b862a487b9350f15f65b8d8ee" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } } peers { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } } peers { permanent_uuid: "638775196708415f931cd7d425669495" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } } }
I20260812 16:36:49.206136 14075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:36:49.206792 14027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "09cfa07b862a487b9350f15f65b8d8ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cfa07b862a487b9350f15f65b8d8ee" member_type: VOTER last_known_addr { host: "127.19.238.126" port: 46191 } } peers { permanent_uuid: "5d64fae61919498898d0fffb5e835cbb" member_type: VOTER last_known_addr { host: "127.19.238.125" port: 36973 } } peers { permanent_uuid: "638775196708415f931cd7d425669495" member_type: VOTER last_known_addr { host: "127.19.238.124" port: 35359 } } }
I20260812 16:36:49.206888 14027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee [sys.catalog]: This master's current role is: LEADER
I20260812 16:36:49.207347 14091 mvcc.cc:205] Tried to move back new op lower bound from 7317719487174782976 to 7317719487128838144. Current Snapshot: MvccSnapshot[applied={T|T < 7317719487174782976}]
I20260812 16:36:49.207934 14065 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:36:49.208647 14065 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:36:49.214949 14065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09cfa07b862a487b9350f15f65b8d8ee: Generated new TSK 0
I20260812 16:36:49.215150 14065 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 16:36:49.240983 14024 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:49.239118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40804 (local address 127.19.238.124:35359)
0812 16:36:49.239170 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.239173 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.240238 (+ 1065us) server_negotiation.cc:407] Connection header received
0812 16:36:49.240351 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.240353 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.240365 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.240398 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.240899 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40804: BlockingRecv error: recv got EOF from 127.0.0.1:40804 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 16:36:49.241782 13940 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:49.239554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38034 (local address 127.19.238.125:36973)
0812 16:36:49.239597 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.239599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.241249 (+ 1650us) server_negotiation.cc:407] Connection header received
0812 16:36:49.241354 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.241355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.241367 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.241399 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.241715 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38034: BlockingRecv error: recv got EOF from 127.0.0.1:38034 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 16:36:49.242560 13851 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:49.239946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43230 (local address 127.19.238.126:46191)
0812 16:36:49.239991 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.239993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.242022 (+ 2029us) server_negotiation.cc:407] Connection header received
0812 16:36:49.242123 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.242124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.242137 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.242168 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.242481 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43230: BlockingRecv error: recv got EOF from 127.0.0.1:43230 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 16:36:49.253331 13940 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:49.252272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38046 (local address 127.19.238.125:36973)
0812 16:36:49.252334 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.252336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.252621 (+ 285us) server_negotiation.cc:407] Connection header received
0812 16:36:49.252756 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.252758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.252784 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.252818 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.253265 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38046: BlockingRecv error: recv got EOF from 127.0.0.1:38046 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 16:36:49.256759 13940 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:49.255800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38058 (local address 127.19.238.125:36973)
0812 16:36:49.255901 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.255902 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.256168 (+ 266us) server_negotiation.cc:407] Connection header received
0812 16:36:49.256294 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.256295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.256312 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.256346 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.256701 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38058: BlockingRecv error: recv got EOF from 127.0.0.1:38058 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 16:36:49.260782 13940 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:49.259978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38070 (local address 127.19.238.125:36973)
0812 16:36:49.260014 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.260015 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.260255 (+ 240us) server_negotiation.cc:407] Connection header received
0812 16:36:49.260354 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.260355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.260366 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.260396 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.260728 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38070: BlockingRecv error: recv got EOF from 127.0.0.1:38070 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 16:36:49.266848 13940 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:49.266014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38078 (local address 127.19.238.125:36973)
0812 16:36:49.266053 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.266055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.266294 (+ 239us) server_negotiation.cc:407] Connection header received
0812 16:36:49.266396 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.266397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.266409 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.266439 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.266776 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38078: BlockingRecv error: recv got EOF from 127.0.0.1:38078 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 16:36:49.277220 13940 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:49.276253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38086 (local address 127.19.238.125:36973)
0812 16:36:49.276299 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.276302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.276580 (+ 278us) server_negotiation.cc:407] Connection header received
0812 16:36:49.276691 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.276692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.276705 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.276735 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.277143 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38086: BlockingRecv error: recv got EOF from 127.0.0.1:38086 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 16:36:49.295706 13940 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:49.294699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38100 (local address 127.19.238.125:36973)
0812 16:36:49.294749 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.294751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.295071 (+ 320us) server_negotiation.cc:407] Connection header received
0812 16:36:49.295185 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.295186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.295199 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.295231 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.295645 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38100: BlockingRecv error: recv got EOF from 127.0.0.1:38100 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 16:36:49.330215 13940 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:49.329241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38114 (local address 127.19.238.125:36973)
0812 16:36:49.329288 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.329291 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.329575 (+ 284us) server_negotiation.cc:407] Connection header received
0812 16:36:49.329688 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.329689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.329702 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.329734 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.330150 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38114: BlockingRecv error: recv got EOF from 127.0.0.1:38114 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 16:36:49.396456 13940 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:49.395739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38118 (local address 127.19.238.125:36973)
0812 16:36:49.395857 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.395860 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.395892 (+ 32us) server_negotiation.cc:407] Connection header received
0812 16:36:49.395958 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.395959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.395975 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.396044 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.396372 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38118: BlockingRecv error: recv got EOF from 127.0.0.1:38118 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260812 16:36:49.526808 13940 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:49.525891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38134 (local address 127.19.238.125:36973)
0812 16:36:49.525997 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.525999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.526111 (+ 112us) server_negotiation.cc:407] Connection header received
0812 16:36:49.526226 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.526227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.526248 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.526318 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.526740 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38134: BlockingRecv error: recv got EOF from 127.0.0.1:38134 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260812 16:36:49.785267 13940 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:36:49.784419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38138 (local address 127.19.238.125:36973)
0812 16:36:49.784486 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:49.784488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:49.784524 (+ 36us) server_negotiation.cc:407] Connection header received
0812 16:36:49.784630 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:49.784631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:49.784643 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:49.784690 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:36:49.785168 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38138: BlockingRecv error: recv got EOF from 127.0.0.1:38138 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 16:36:50.163169 14056 catalog_manager.cc:1259] Loaded cluster ID: 4a770b999f684c80ae49afb12e484e97
I20260812 16:36:50.163254 14056 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495: loading cluster ID for follower catalog manager: success
I20260812 16:36:50.164247 14056 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495: acquiring CA information for follower catalog manager: success
I20260812 16:36:50.164582 14056 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 638775196708415f931cd7d425669495: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:36:50.205812 14090 catalog_manager.cc:1259] Loaded cluster ID: 4a770b999f684c80ae49afb12e484e97
I20260812 16:36:50.205921 14090 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb: loading cluster ID for follower catalog manager: success
I20260812 16:36:50.206902 14090 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb: acquiring CA information for follower catalog manager: success
I20260812 16:36:50.207229 14090 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5d64fae61919498898d0fffb5e835cbb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:36:50.301435 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:50.298548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38142 (local address 127.19.238.125:36973)
0812 16:36:50.298661 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:50.298664 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:50.298732 (+ 68us) server_negotiation.cc:407] Connection header received
0812 16:36:50.298802 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:50.298803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:50.298876 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:50.298981 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:36:50.299147 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:50.299659 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:50.300475 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:50.300749 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:50.300824 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:50.301041 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:36:50.301196 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:36:50.301289 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:50.301306 (+ 17us) server_negotiation.cc:299] Negotiation successful
0812 16:36:50.301330 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":76,"threads_started":1}
I20260812 16:36:50.306202 14174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:50.303499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40816 (local address 127.19.238.124:35359)
0812 16:36:50.303630 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:50.303632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:50.303642 (+ 10us) server_negotiation.cc:407] Connection header received
0812 16:36:50.303733 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:50.303734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:50.303792 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:50.303928 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:36:50.304052 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:50.304552 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:50.305300 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:50.305530 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:50.305607 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:50.305791 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:36:50.305940 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:36:50.306034 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:50.306074 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 16:36:50.306097 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":57,"threads_started":1}
I20260812 16:36:50.311487 14184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:50.308415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43246 (local address 127.19.238.126:46191)
0812 16:36:50.308513 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:50.308515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:50.308526 (+ 11us) server_negotiation.cc:407] Connection header received
0812 16:36:50.308778 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:50.308780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:50.308835 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:50.308919 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:36:50.309097 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:50.309650 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:50.310554 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:50.310797 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:50.310857 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:50.311097 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:36:50.311216 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:36:50.311278 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:50.311359 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 16:36:50.311385 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":49,"threads_started":1}
I20260812 16:36:50.311617 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:50.308256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38158 (local address 127.19.238.125:36973)
0812 16:36:50.308306 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:50.308307 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:50.308636 (+ 329us) server_negotiation.cc:407] Connection header received
0812 16:36:50.308706 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:50.308707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:50.308718 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:50.308774 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:36:50.308923 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:50.309650 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:50.310830 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:50.311081 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:50.311142 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:50.311331 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:36:50.311369 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:36:50.311419 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:50.311512 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 16:36:50.311539 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 16:36:50.311744 14174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:36:50.308313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40820 (local address 127.19.238.124:35359)
0812 16:36:50.308355 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 16:36:50.308357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 16:36:50.308508 (+ 151us) server_negotiation.cc:407] Connection header received
0812 16:36:50.308599 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:36:50.308600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:36:50.308613 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:36:50.308670 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:36:50.308799 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:50.309595 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:50.310801 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:36:50.311103 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:36:50.311181 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:36:50.311365 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:36:50.311399 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:36:50.311429 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0812 16:36:50.311635 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 16:36:50.311663 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 16:36:50.313627 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 13769
I20260812 16:36:50.322165 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 13855
I20260812 16:36:50.328786 20409 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt79009/build/release/bin/kudu with pid 13944
2026-08-12T16:36:50Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2957 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5879 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (283157 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST